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ABSTRACT 



This study examined college students' perceptions of course 
Web sites as an instructional resource for classroom-based courses. The focus 
was on identifying functions on the sites that students perceived as 
supporting and fostering their learning experiences. Subjects were 142 
students responding to a 60-item questionnaire and open-ended questions. 
Findings indicated an overall positive perception of the quality and 
usefulness of the course Web sites. On average per typical semester week, 64 
minutes were spent on conducting searches on the Web, 40 minutes on 
downloading and printing material, and 34 minutes on communicating with 
faculty and teaching assistants. Highest ratings of instructional quality 
were on the visual appeal and readability of sites and the importance of the 
material on the site. Lowest were on the clarity and purposeful introductions 
to each segment on the site, clarity of the connection of each new section on 
the site with course objectives, and general taste in color of the pages. 
Highest ratings for perceived usefulness were on the use of visuals to recall 
or present new information and the opportunity to ask questions online. 

Lowest were the use of links to review/recall prerequisite material and 
instruction on how to navigate the site. Greatest barriers to use were access 
to computers and to Web site addresses, perceived in adequacy of their 
Internet skills, motivation to use the site, and time constraints. Greatest 
facilitators of use were guidance, quality of content, availability of 
material, access to material, faculty, peers, teaching assistants, experts, 
and ease of communication. Overall impact of course Web sites was time saving 
qualities, 24 -hour accessibility to resources, facilitating preparation for 
class, and increased understanding of class expectations and objectives. 

There appeared to be a negative relationship between residential distance 
from campus and perceived usefulness of sites and a possible relationship 
between courses and students' perceived instructional quality on functions 
related to clarity of purpose and objectives. Also, there appeared to be a 
general lack of motivation to use the sites, possibly due to their lack of 
mandatory use and what students reported as a lack of incentive to use them 
for specific course requirements. (AEF) 
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The purpose of this study was to examine college students' perceptions of course 
Web sites as an instructional resource for classroom-based courses as one case study of a 
private Research II university faculty initiative to integrate WWW technology into 
campus-based courses. The focus of the study was on the identification of functions on 
the course Web sites that students perceived as supporting and fostering their learning 
experiences. 

One hundred and forty two students responded to a 60-item questionnaire on the 
time spent using their respective course Web sites, their perceptions of Web site 
instructional quality and usefulness. Students also responded to open-ended questions on 
their perceived barriers to and facilitators of using the course Web sites and their overall 
perceived impact of the WWW on their learning experiences. Nine out 174 course Web 
sites offered in the fall 1998 were selected based on a screening test using a 30-item 
instrument. In addition to the Student Web site Perception Survey (SWPS) and the Web 
site Instructional Design Checklist (WIDEC), a third instrument was developed and 
administered to participating faculty for triangulation purposes when interpreting the 
data. 

Study findings indicated an overall positive perception of the quality and 
usefulness of the course Web sites. In terms of time spent on given functions, on average 
per typical semester week, 64 minutes was spent on conducting searches on the WWW, 
40 minutes on downloading and printing material, and 34 minutes on communicating 
with faculty and teaching assistants. The highest ratings of instructional quality were on 
the visual appeal and readability of Web sites and the importance of the material on the 
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Web site. The lowest ratings of instructional quality were on the clarity and purposeful 
introductions to each segment on the Web site, the clarity of the connection of each new 
section on the Web site with course objectives, and the general taste in color of pages. 

The highest ratings for perceived usefulness were on the use of visuals to recall or present 
new information and the opportunity to ask questions online. The lowest ratings were the 
use of links to review/recall prerequisite material and instruction on how to navigate the 
Web site. 

Students reported that the greatest barriers to use were access to computers and to Web 
site addresses (URLs), perceived inadequacy of their Internet skills, motivation to use the 
course Web site, and time constraints. The greatest facilitators of use were guidance, 
quality of content, availability of material, access to material, faculty, peers, teaching 
assistants, and experts, and ease of communication. Overall impact of course Web site 
was its time saving qualities, 24-hour accessibility to resources, facilitating preparation 
for class, and increased understanding of class expectations and objectives. Finally, there 
appears to be a negative relationship between residential distance from campus and 
perceived usefulness of course Web sites and a possible relationship between courses and 
students’ perceived instructional quality specifically on functions related to clarity of 
purpose and objectives. In addition, there appears to be a general lack of motivation to 
use the course Web sites, possibly due to their lack of mandatory use and what students 
reported as a lack of incentive to use them for specific course requirements. 
Recommendations and suggestions for further research are also provided. 
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CHAPTER ONE 

Problem Statement 



As intellectual capital becomes recognized as the asset of the future, universities mil 
begin to realize that their most important asset is the student body and the university workforce, 
not only the faculty, but the entire workforce. The university is after all a learning environment, 
and the capital of the university is the knowledge, skill, and intellect of the work force. The 
product is an essential service, and the clients are the students (also often members of the 
worlforce), government, community, corporations, and the nation. (Mauch, 1998, p. 12). 

1.1 Introduction 

Dissatisfaction with the educational status quo, both from inside and outside the 
university, has increased the pressure on higher education to reevaluate its traditional 
educational practices. Higher education is facing the challenges of the rapid advancement 
of information technology, increased accountability for quality education, competition from 
online institutions, rising costs, and changing student population (Ehrmann, 1995; Guskin, 
1994; Handy, 1998; Owston, 1997). 

Consequently, some universities and colleges are considering several restructuring 
solutions. Some of these solutions target educational practices: modifications in mission 
statements urging better preparation of students for work and life skills (Handy, 1998); 
curricula re-design with emphasis on measurable learning outcomes (Diamond, 1997); a re- 
definition of pedagogical goals toward a more learner-centered pedagogy (Collis, 1998); 
and heavy investment in technological innovations; such as the Internet (Owston, 1997). 

Of these solutions, the latter has received much attention in higher education literature. 
Administrators have high expectations that the potential of the Internet can be realized to 
better serve higher education clientele (Duchastel, 1997) and, thereby, address some of its 
restructuring concerns (Ehrmann, 1997a; Guskin, 1994; Owston, 1997). Reasons for such 
expectations arise from several factors. 
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First, the Internet’s communication capabilities and its ability to transmit and store 
vast amounts of information hold great promise for the restructuring of teaching and 
learning practices. Already, two of its most popular services. Electronic mail (e-mail) and 
the World Wide Web (WWW/ Web') are changing teaching and learning practices since 
“teacher and text” are no longer perceived as the only resources available to learners. 
Today, e-mail allows for access to global peers, colleagues, experts, and researchers. 
Similarly, the WWW brings a variety of multi-media resources to the fingertips of students 
at a speed and quality of transmission never experienced before through networked 
computer technology. 

The 1999 Campus Computing Project's annual national survey of information 
technology in higher education reported that many higher education institutions were 
providing more services through the WWW. For example, more than half (69.5%) of the 
institutions responding provided online undergraduate applications on their Web sites. 
More than three-fourths (77.3%) made the course catalog available online. A quarter 
(25%) of the institutions made library-based course reserve readings available online and 
almost a half (46.5%) offered one or more full college courses online via the Internet and 
the WWW (Campus Computing Project, 1999). 

Second, the Internet allows for greater adaptability of instructional methods to 
different learning environments making it ideal for distributed learning systems capable of 
accommodating the academic needs of a diverse adult student population. 

In particular, adult students with work and family responsibilities, who do not have 
the luxury of attending school on campus, are able to have access to faculty and resources 
provided by the traditional educational institutions through this medium. Trend setting 

' The WWW and Web will be used interchangeably throughout the document. 
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models, such as for-profit universities, virtual universities, and corporate partnerships are 
taking advantage of this technological potential. Examples of the for-profit models are the 
University of Phoenix that has no full-time or tenured faculty and the International 
University that has a mere seven employees and 600 students (Oppenhimer, 1997). 
Examples of the corporate partnerships are Motorola and Northwestern University, 
American Express and Rio Salvo Community College, and NYNEX and State University 
of New York (SUNY) (Mauch, 1998). 

The co-operation between universities and businesses is predicted to become 
essential in the near future to provide and ensure modem and relevant education and 
training (Mauch, 1998). Already, several of these corporations are granting degrees 
themselves, or in connection with universities and colleges. Moreover, virtual universities 
have no physical campus and, with faculty located world-wide, they can provide increasing 
accessibility to learning and to a much more diverse group of academic specialists 
(Harasim, Clavert, & Groeneboer, 1997; Owston, 1997). The above mentioned 
organizations make strong claims that they are taking advantage of the new technologies 
and are meeting students' educational needs (Burdman, 1998a). 

Third, traditional classrooms in higher education have been criticized for inadequate 
training of students for the work place. The boredom, seeming lack of personal attention, 
outdated class material, and inadequate sensitivity to a diverse population are some of the 
issues that have been questioned (Gardiner, 1997; Handy, 1998). 

To further complicate matters, it has been argued that many students graduate 
without mastering core skills (Diamond, 1997). Meanwhile, state-of-the-art technologies 
are being adopted by corporations who are spending billions of dollars annually on formal 
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training to make curricula and teaching methods more relevant, interesting, and modem 
(Mauch, 1998). Such possibilities are forcing the traditional establishment and 
administration of higher education to re-think and re-stmcture some of its teaching and 
learning practices to be able to compete within a changing supply and demand equation. 

Consequently, the above-mentioned pressures and challenges leave few options for 
higher education but to jump onto the technological bandwagon (Sherritt & Basom, 1997). 
Similarly, Rossner and Stockley (1997) summed up the situation as one of: 

...increasing public access to information technology, government funding 
priorities for initiatives that support the development of distributed learning 
networks, increasing competition for students who no longer need to be present on a 
university campus, and the need to provide cost-effective, high quality education to 
greater numbers of students, (p. 333) 

It is not surprising, therefore, that an intellectual debate has been brewing in higher 
education. Administrators have to contend with the pressure to integrate technology as a 
means of attracting student applications and faculty either rejecting its imposition onto their 
teaching practices or seek support for technology integration (Campus Computing Project, 
1999), rewards, and recognition for the time and effort that goes into such efforts. Green 
(1999) reported that the 1999 survey of college faculty from UCLA's Higher Education 
Research Institute (HERI), which provides the first large-scale data on some aspects of the 
way college faculty use — and do not use — IT resources documented that “keeping up with 
information technology" is a major source of stress for fully two-thirds of the professoriate. 
(p3). 
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In the midst of all these re-structuring efforts, it is the students who are facing the 
reality of a changing educational experience. Very few educational scholars have paid 
attention to the experience of the students who already use the WWW. In addition to 
students who rely on distance education served by the proliferation of online curricula, 
campus-based students are also becoming attracted to the "online" experience, not only for 
entertainment and personal purposes, but also for academic purposes. For example, at the 
University of Colorado at Denver, 80% of students enrolled in one university online course 
intended to boost its adult education program, were on-campus undergraduates (Burdman, 
1998a). It was reported that the students preferred the convenience of logging on at 
midnight to waking up for an 8:00 a.m. class. However, even students who do not register 
for online courses may find themselves, on the first day of class, being directed to a course 
Web site intended to "support" and "enhance" their traditional course. Through this 
medium, students are told that they can log on, download class notes, view important class 
resources, complete assignments, receive feedback, and even take a quiz, all at their own 
convenience. 

However, they are not necessarily informed that this convenience comes at a price. 
Many students find that if they do not own a computer, they probably have but two choices 
either wait in line in over-crowded computer clusters or purchase the computer hardware 
necessary to fully participate online. Further, this proclaimed "convenience" may be 
hindered by additional costs accrued to related services. 

In fact, the 1996 annual Campus Computing Project survey revealed that a third of 
all campuses (31.8%) required a student technology fee (Campus Computing Project, 
1996b). In addition to concerns for cost and time constraints, the quality of students 
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learning experiences may be compromised by inappropriate use of WWW technology. 
These changes should be documented from a student’s perspective and not restricted to 
educators’ and administrators’ so that technology integration efforts better serve teaching 
and learning practices. 

1.2 Background and Problem 

Though greeted with enthusiasm by the university community at large, 
technological integration efforts remain controversial. Opponents caution against hasty and 
ad hoc technology integration into the daily practices of teaching and learning, specifically 
without taking into consideration faculty and student willingness and cooperation in the 
integration efforts (Noble, 1998; Schrage, 1998; Turoff, 1997; Young, 1998). Because the 
integration of various technologies into teaching and learning practices arrives with such 
great expectation (Brown, 1998), we may be blind to the consequences of those very 
expectations. For example, students may expect that, because they e-mail faculty at all 
times of the day that, faculty will respond in a like manner. Administrators may expect 
that WWW technology can be designed and maintained easily and so they may not take 
technological integration efforts into account when making decisions about faculty reward, 
tenure, and promotion (Campus Computing Project, 1996b). Faculty may be expected to 
keep abreast of software innovations, in addition to their scholarly specialties, to enhance 
their teaching practices. Rarely appreciated is the time and effort that both faculty and 
students invest in learning how to use various technologies. 

Over the past two years, we have witnessed controversial reactions to WWW 
technology integration efforts. For example, in the summer of 1998, 850 faculty members 
from the University of Washington signed a letter to the Governor protesting the possibility 
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of instructors being replaced by computerized teaching tools (Monaghan, 1998). A year 
earlier, during the summer of 1997, the University of California at Los Angeles (UCLA) 
created Web pages for nearly every undergraduate course, although it did not mandate their 
use (Young, 1998). Apparently, this initiative was launched during the summer when 
many faculty were away, despite the formal student recommendation against it (Noble, 
1998). Faculty complained that this type of approach was ‘heavy-handed’ and argued that 
not every course benefits from the Web (Young, 1998). Furthermore, some faculty 
members considered such actions by the administration as an infringement on faculty 
academic freedom by putting an unwanted teaching tool in their courses (Young, 1998). 
Meanwhile students, who were required to pay fees for such Web sites, argued that many 
of these Web sites were not worth the fee they were paying (Young, 1998). Earlier, during 
the spring of 1997, the faculty at York University, Canada’s third largest campus, ended a 
two-month-long strike after having secured a contractual protection against that type of 
administrative action that was taken later by UCLA (Noble, 1998). Noble, leader of the 
faculty strike at York University, stated that the strike was in response to: 

...unilateral administrative initiatives in the implementation of instructional 
technology, the most egregious example of which was an official solicitation to 
private corporations inviting them to permanently place their logo on a university 
online course in return for $10,000 contribution to courseware development. (1998, 
P-2) 

Noble (1998) maintained that, in a fashion similar to UCLA's, “the initiatives of the York 
administration in the deployment of computer technology in education were taken without 
faculty oversight and deliberation, much less student involvement” (p.3). 
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Significantly, at both UCLA and York, the somewhat presumed 'satisfied and 
happy' students have given clear indication that they are not so optimistic about the 
prospect of a high-tech academic future (Noble, 1998). These students recommended 
against the initiative at UCLA and at York, provided support to striking faculty, and 
launched their own investigation into the commercial, pedagogical, and ethical implications 
of online educational technology (Noble, 1998). In spite of that, UCLA officials were 
highly optimistic regarding students’ enthusiasm about the high-tech agenda. Noble 
described the consequences as follows: 

To date, however, there has been no such demand on the part of students, no 
serious study of it, and no evidence for it. Indeed, the few times students have been 
given a voice, they have rejected the initiatives hands down, especially when they 
were required to pay for it. (p. 10) 

While the UCLA initiative was met with much resistance, there was a positive side 
to the story reported by Young (1998). For example, several professors welcomed the 
program with the rationale that it gave them the support they needed to try new things, 
enhanced the computer level of competency among colleagues and students, and allowed 
for easier distribution of materials to students. Professors of the sciences, it was reported, 
expressed more comfort with the WWW than those in the humanities. Students also 
expressed some positive reactions to the program. For example, several students liked the 
convenience of accessing information during early morning hours and receiving timely 
feedback from their course instructors. Other students appreciated the quality of 
information made available in various modes of presentation be it pictures and examples 
for a particular unit or program information on a given course to be offered in the following 
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semester. Young (1998) also reported that a survey of 4,000 students conducted by the 
university in the fall of 1997 revealed that 60% of students felt that the WWW pages had 
increased their interaction with professors. It is not surprising then that UCLA officials, as 
reported in a different context by Noble (1998), “would insist that faculty make fuller use 
of the Web site technology in their courses” (p. 10). 

In sum, the Internet is being heralded as a versatile medium to support learner- 
centered educational practices. It is being promoted as a technological innovation that can 
reduce mundane time-consuming tasks of faculty workload. However, there is evidence 
for discontentment and controversial opinions regarding the above from both faculty and 
student bodies that only time and empirical research could serve to dispel. This study seeks 
to voice students' opinions based on the belief that in a student-centered educational 
institution such as the case represented in this study, students should participate in the 
decision-making process related to WWW technology integration efforts by voicing the 
impact of the changes they confront. 

1.3 Study Purpose and Research Questions 

The purpose of this study was to examine college students' perceptions of their 
course Web sites as an instructional resource for classroom-based teaching at Syracuse 
University, a private university classified as a Research II university (Carnegie 
Classification - Directory of Higher Education, 2000) located in the northeast of the United 
States. The focus of the study was on the identification of functions on their course Web 
sites that students perceived as supporting and enhancing their learning experiences. This 
study therefore explored the time students spent using the course Web sites, as well as their 
perceptions of the instructional quality and usefulness of these functions. Study findings 
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also contributed to closing gaps in empirical research on Web-based instruction pertaining 
to student use of various Web site functions for instructional purposes, the usefulness and 
quality of these functions, and the conditions that promote or inhibit learning using Web 
technology. Since these issues from a student perspective were hardly examined in prior 
research the following questions were posited to achieve the purpose of the study: 

1) How did students allocate their time using their course Web sites? 

2) What was the quality of the course Web sites according to the students? 

3) How useful did students perceive their course Web sites? 

4) What demographics and characteristics influenced students' use and perceptions of 
quality and usefulness of course Web sites? 

5) What were some student reported barriers to and facilitators of use of the course Web 
sites and overall impact of the WWW on students' learning experiences? 

1.4 Theoretical Framework 

A conceptual framework was developed to ground and guide the research 
conducted in this study. Figure A illustrates this conceptual framework. There is evidence 
from research findings that certain principles exist in most successful teaching and learning 
practices (Chickering and Gamson, 1987, 1991). Second, there is evidence in the literature 
that the application of instructional design principles to teaching and learning practices 
leads to more effective learning (Gagne 1975, Gagne, Briggs, & Wager, 1992). One of the 
important measures of effective instruction is how students feel about the instructional 
process (Dick & Reiser, 1989). Based on the above premises, one can make the 
assumption that a well-designed instructional Web site can support and foster good 
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teaching and learning practices and therefore become a valuable instructional resource for 
both faculty and students. 

This framework illustrated in Figure A depicts the scope and theoretical context of 
the study. In this framework, the two foci are (a) the course Web sites and (b) students’ 
time on task and their perceptions of the quality and usefulness of Web site functions 
(outcome measures). The course Web sites sampling pool is studied using two sets of 
theoretical principles. The first is Gagne, Briggs, and Wager's (1992) set of nine events of 
instruction (hereby referred to as the nine events). Gagne et al.’s (1992) nine events of 
instruction are considered essential elements to effective instruction (Richey, 1996; Richey, 
2000; Ritchie & Hoffrnan, 1996; Smith & Ragan, 1996). 
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The second set of principles is Chickering and Ehrmann (1996) conceptualization 
of the seven principles developed by Chickering and Gamson (1987, 1991) and placed in a 
technology-based context whereby various technologies are used as a lever to promote 
these good practices (hereby referred to as the seven principles). All principles but two 
were relevant to the scope of the study. Principle 6, communicating high expectations and 
principle 7, respecting diverse ways and talents, while highly important, were believed 
beyond the scope and timelines of the study and were therefore excluded. 

Moreover, functions presented by the course Web site were measured using the two 
sets of principles mentioned above. They were also presented to students as the criteria of 
evaluation of their course Web sites and were the basis for item development in the study 
instruments for data collection (see section 2.6, 3.6.2, and Appendix B and C for 
instruments). Figure A further illustrates the focus of the study as four outcomes measures; 
(a) use as indicated by time on task; (b) usefulness; (c) quality; and (d) factors that 
influenced the first three outcomes. It is important to note that the author is aware that this 
conceptual framework is by no means a comprehensive or accurate representation of the 
reality. It is only intended to simplify the complexity of the inter-relationships between the 
variables under study. These measures are studied within two units of analysis: (a) the 
course Web sites and (b) student characteristics and demographics. Demographics and 
characteristics include: student Internet expertise, computer accessibility, years of 
accessing the WWW, GPA, gender, academic discipline, reasons for taking the course, 
proximity of residence to campus clusters, modem speed and connectivity, among others in 
Figure A. These variables were selected based on narrative data gathered from ten one- 
hour interviews with students taking one of the courses represented in the sample. These 
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variables also draw upon the learning environment at Syracuse University and there are 
relevant contextual factors to examine to support and explain findings from the quantitative 
data. The theoretical framework depicted in Figure A is further discussed in section 2.6 in 
chapter two. 

Traditionally, in the field of educational technology, demographics variables have 
been examined in relation to technology use to test hypotheses related to learning 
achievement. This has lead to ongoing debates about significant versus non significant 
findings in comparative studies between online education and traditional education on 
student achievement measures, for example, studies by Chen, Lehman, & Armstrong 
(1991); Despain (1997); Efendioglo & Murray (2000); Fulmer, Hazzard, Jones, & Keene 
(1992); Hammond (1997); Martin & Rainey (1993); Morrissey (1998); Nesler & Lettus 
(1995); Schutte (1998), and Stinson & Claus (2000). In this study, however, the purpose of 
studying these variables is simply to situate the study findings in a familiar context where 
the phenomenon occurs, i.e., within one larger case study of a private university with those 
particular student demographics and characteristics. The second purpose is to provide 
possible insights in interpreting the findings rather than to add to the significant versus non- 
significant debate or to test the hypothesis that "increased" learning has occurred using the 
new technology. 

In summary, Gagne’s et al. (1992) nine events of instruction and Chickering and 
Ehrmann's (1996) seven principles of good teaching and learning practices using 
technology as a lever were selected to serve as a conceptual framework and a source of 
theoretical grounding for the study to be drawn upon in the construction of outcome 
measures of what students value about the WWW when used to support classroom teaching 
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and learning functions. Specifically, the theoretical framework served to operationalize 
three dimensions used in this study: Time Spent as an indicator of course Web site use, 
Instructional Quality, and Usefulness. These dimensions guided instrument development 
to be presented in chapter three. Moreover, the findings should add to a growing body of 
literature on optimal use and integration of WWW technology to improve teaching and 
learning experiences. Second, the findings should provide empirical research for the 
applicability of these two theoretical frameworks in a Web-based context. Third, the 
findings should provide insights into functions, when present, are perceived valuable to a 
student using this site as a resource to enhance his/her learning efforts. 

Various instructional design models and theories were considered for adoption by 
this study. Given that the focus of the study was to identify what Web site functions 
students perceived as supporting and enhancing their learning experience, a theoretical 
model that explicitly defines the elements of instructional sequencing and support was 
required. A model was determined necessary to suit the purpose of developing some 
evaluation criteria for Web-based instruction given the difficulty in finding explicit theory- 
driven instructional Web site evaluation criteria in the related literature. 

Several relevant traditional and constructivist models were examined such as 
Madeline Hunter's Elements of Lesson Design (Hunter, 1 967), Bloom 's Taxonomy of the 
Cognitive Domain (Bloom, Engelhart, Furst, Hill, &Krathwohl, 1956), the Direct 
Instruction Model (Joyce, Weil, & Showers, 1994, as cited in Driskell, 1999), the 
Constructivist Learning Environments Model (Honebein, 1996, as cited in Driskell, 1999), 
and the Web-based Instructional Design Model (El-Tigi & Branch, 1997). However, none 
were found to be as elementary, encompassing, generative, and applicable to the 
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development of a set of evaluation criteria as the nine events (see section 3.5 for instrument 
development). 

1.5 Significance of the Topic and Study 

Educational institutions require better information about the consequences of 
current educational uses of various technologies (Ehrmann, 1997a). Research findings 
contribute to a growing body of research that supports decisions on WWW technology 
integration specific to campus-based classes. As more and more organized instruction is 
delivered on the WWW, and as students gain more access to the Internet, its effective 
utilization and integration into their learning experiences becomes more crucial. 

The rapid growth of Internet use in higher education augmented by the various 
economic, political, and social pressures on higher education to integrate networked 
technologies in the classroom are the primary driving forces behind the changes in today’s 
learning environment. This learning environment must cater to global job market skills and 
knowledge that depend heavily on cutting edge technologies for communication and trade 
(Guskin, 1994; Romiszowski, 1997; Owston, 1997). One rationale is that one key to 
improved learning with the Web lies in effective utilization of this medium in teaching and 
learning (Owston, 1997). 

Similarly, as we witness changes in student demographics, occupational 
requirements, the rising tuition costs and decreasing student enrollments in some traditional 
campus-based courses, the argument for distributed learning environments, in the form of 
Web technology for instructional support purposes, grows stronger. The Internet emerges 
as one technological solution to an educational challenge. It also forces traditional 
education toward re-engineering and re-structuring itself to meet the needs of its clientele. 
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This study seeks to give voice to a large segment of higher education's clientele, namely the 
students. 

Delivering instruction using the WWW, like any other instructional design effort, 
requires detailed analysis of user interaction, appropriate selection and optimization of 
various media capabilities, and implementation of sound instructional design principles in 
order to provide high pedagogic value to students (Gagne, Briggs, & Wager, 1992; Dick & 
Carey, 1996). This necessitates a closer examination of the student population that will be 
most affected by technology integration efforts. In higher education settings, students are 
the primary consumers and users of Web technology and the primary target audience of 
course Web sites. Hence, documentation of their opinions, attitudes, and first-hand 
experiences with instructional course Web sites constitutes significant feedback to the 
instructional design effort and the establishment of good teaching and learning practices. 

Exploring some dimensions of the current use of instructional course Web sites 
serves to contribute to research studies that address fundamental questions that loom large 
in the literature; for example, questions like the ones posited by Windschitl (1998) on the 
pedagogic value of the WWW asking: (a) Are WWW-based practices helping students, and 
if so, how?" and (b) How is the introduction of this technology changing pedagogical 
practices?" or evaluation questions on Web-based instruction such as those identified by 
Trochim (1996): (a) How are students using course Web sites? (b) What features are most 
effective for learners? and (c) How best can we evaluate learning features contained on 
Web sites? 

Hence, this study focused on examining students’ perceptions of course Web sites 
as an instructional resource, specifically students reported time on task, and the usefulness 
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and instructional quality of various functions. Students also reported on the WWW impact 
on learning as well as barriers to and facilitators of course Web site use. In addition, 
conditions that influenced these perceptions were also explored with particular emphasis on 
some student characteristics and demographics. Indeed, as far back as 1995, Schrum 
stated, “It is timely to study these (online) courses now so that educators can take a strong 
leadership role in assuring that courses are pedagogically sound, organizationally strong, 
and institutionally supported” (1995, p. 1). Schrum, also advocated that learners be given a 
larger role in the design of online courses. One way to contribute to this goal would be to 
take into account students’ perspectives on instructional solutions using functions 
supported by Web technology that they consider useful and facilitative of their learning 
experiences. 

1.6 Dissertation Organization 

This dissertation is organized into six chapters, with this chapter as the introductory 
chapter stating the problem, purpose, focus and theoretical grounding of the study. Chapter 
two is a review of relevant literature covering current controversy about use of technology 
in higher education and justification for such use. The chapter also describes the 
technology and some of its perceived benefits and limitations. Chapter three outlines the 
methodology for the study and includes a brief contextual overview about WWW 
technology integration at Syracuse University. Descriptions are given of the participants, 
design methods, data collection strategies, variables included in the study, and statistical 
and qualitative analysis procedures employed. Chapter four describes the design, 
development, and validation of the Student’s Web site Perception Survey (SWPS). 

Chapter five presents the results of the study in both narrative and tabular formats. The 
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final chapter, chapter six, summarizes the study and its limitations. Methodological, 
theoretical, and practical implications are discussed. In addition, conclusions drawn from 
this research as well as suggestions for future research bring this thesis report to closure. 

1.7 Chapter Summary 

This study investigated the extent to which higher education students valued their 
course web sites and the factors that influenced those perceptions. The study draws upon a 
theoretical framework for study design and implementation. Findings from the study 
provide empirical data to better inform decision-makers with regard to current teaching and 
learning experiences when using WWW technology as an instructional resource to support 
classroom based teaching. [A glossary of terms is provided in Appendix A]. Figure C 
provides a pictorial overview of the study is located at the back of this document for 
convenience (see page 229). Chapter two presents relevant research including empirical 
research findings on the presence of the IntemetAVWW in higher education, the history of 
the WWW, a definition and typology of Web-based instruction (WBI) including empirical 
research on WBI. Finally a discussion of the nine events of instruction and the seven 
principles of good teaching and learning practices brings the chapter to closure. 
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CHAPTER TWO 

Review of Relevant Literature 

“It is not simply a matter of ‘using ’ new technology for old familiar 
purposes, but of finding that those purposes change, that the way we think and feel 
about them change, that new purposes and new needs come into being that barely 
existed before’’ (Burbules, 1996, p.21). 

2.1 Introduction 

The chapter outlines various perspectives related to both the challenges and the 
possibilities that can be afforded by the Intemet/WWW in a teaching and learning 
environment. The chapter begins with a broad summary of the larger economic and societal 
pressures on higher education to use the Internet. The discussion of the literature continues 
as follows: 

□ The Presence of the Intemet/WWW in Higher Education 

□ The World Wide Web 

♦ History and Characteristics 

♦ The World Wide Web as a Distributed Learning Environment 

□ Web-based Instruction 

♦ A Definition 

♦ A Typology of Course Web sites - WWW Technology Integration 

♦ Highlights on Web-Based Instruction (WBI) Research 

□ Effective Instruction - Theory and Practice 

♦ Gagne’ s Nine Events of Instruction 
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♦ Application of Gagne’s Nine Events of Instruction to Determine the Quality of an 
Instructional Web Site 

♦ The World Wide Web as Lever to Teaching and Learning Practices 

♦ Seven Principles of Good Practice in Undergraduate Education 

□ Chapter Summary 

2.2 The Presence of the Internet/WWW in Higher Education 

The National Center for Education Statistics (NCES) published a statistical analysis 
report on the status of distance education at postsecondary education institutions in a 
survey conducted during winter 1998-1999 (Lewis, Snow, Farris, & Westat, 1999). It was 
reported that 78 percent of public four-year institutions (19% of private 4-year institutions) 
offered distance courses. It was also reported that over 54, 000 college level, credit- 
granting distance courses were offered by 2-year and 4-year postsecondary institutions 
during 1997-1998. By 2001, if the current trend continues, it was anticipated that most 
universities large or small, public or private will likely offer one or more courses online. 
The primary mode of instructional delivery is the Internet, used by 58% of all institutions 
utilizing its asynchronous computer-based capabilities (Lewis, Snow, Farris, & Westat, 
1999). 

The growth in distance education and use of the Internet has made its mark on 
campus-based higher education. This mark becomes evident upon examination of the 
Campus Computing Project (CCP) annual survey findings over a span of six years. CCP 
data are revealing the extent of use of information technology in higher education in the 
United States. The CCP survey is the largest continuing study of the role of information 
technology in US higher education (CCP, 2000). Every year. Campus Computing officials 
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at 550 to 800 two- and four-year colleges and universities participate. The author 
summarized the relevant survey findings pertaining to the status of information technology 
(IT) in higher education over a period of 6 years (1994-1999).^ Earlier reports were 
excluded because they were conducted prior to the presence of the Internet in higher 
education. 

Each year, the report sought to highlight the status of IT in higher education. The 
1994 survey report highlighted IT as moving slowly into the classroom (Campus 
Computing Project, [CCP] 1994). Officials reported that “expanding or enhancing 
‘network access’ remains a top priority “ (CCP, 1994, p. 2). Demand by Internet users 
from traditionally non-technology related disciplines increased. Consequently, expanding 
campus networks remained the top priority. By 1994, 79% of the campuses had Internet 
access. 

The 1995 survey findings highlighted a dramatic growth in the use of IT in college 
courses (CCP, 1996a). In addition to the use of e-mail, CD-ROM, computer simulations, 
the use of the Internet and the WWW also increased. The data also revealed that the use of 
information technology in instruction was breaking into the ranks of mainstream faculty as 
opposed to just the early adopters (Green, 1996a). 

Students’ expectations for integration of technology in their courses, across all 
disciplines, were increasing and so were faculty efforts to integrate technology into the 
curriculum. For example, more than 14% of all campus classes put class materials, such as 
syllabi, on the Web. More than 24% use other Web resources, such as online 
encyclopedias and other Web sites. The data also revealed that “the Internet and WWW 

^ Dr. Kenneth Green, director of the project informed me that a trend report of the past 10 years will be 
published by the end of this year (personal communication on Feb 18*, 2000). 
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access are viewed by a growing number of faculty and students as a core resource as well 
as a basic right, similar to a library card” (Green, 1996a p. 9). Already 71.5% of the 
respondents stated that using Internet resources was very important (CCP, 1996a). 

The 1996 report highlighted instructional integration and user support as presenting 
continuing technology challenges (CCP, 1996b). This was only in response to the growing 
students expectations across all disciplines to integrate technology into learning and 
instructional experience (CCP, 1996b). Interestingly, by 1996, 30.1% of participating 
campuses reported a formal campus plan for the use of the Internet and the WWW in 
instruction. 

The 1997 report highlighted more technology in the syllabus and more campuses 
imposed IT requirements and student fees (CCP, 1997). It was reported that “Electronic 
mail, the Internet, the WWW, and multimedia are increasingly common components of the 
instructional experience for growing numbers of American college students” (CCP, 1997, 
p.l). It was also reported that despite the increase in IT use and the growing number of 
campuses imposing IT requirements on their students, assisting faculty to integrate 
technology into their instruction and providing user support were among the top IT 
challenges confronting higher education institutions (CCP, 1997). 

The 1998 report highlighted colleges struggling with IT planning (CCP, 1998). 
More than 60% of higher education institutions did not have an IT financial plan, under 
50% had a strategic plan for IT, and 60% did not have an instructional plan for using the 
Internet. Again, respondents reported that assisting faculty to integrate technology into 
instruction was the single most important IT issue confronting their institution. (CC, 1998). 
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Most recently, the 1999 report highlighted the continuing challenge of instructional 
integration and user support (CCP, 1999). The challenges of integrating and optimizing 
use of such technologies were continuously reported as growing concerns for many 
educators. This became evident when the 1997-1999 surveys paid closer attention to the 
identification of the single most important IT issue in higher education. The 1997 results 
reported that 29.6% of all institutions considered the number one most important issue to 
provide assistance to faculty efforts to integrate IT into instruction. Similarly, the 1998 and 
1999 results reported a continued increase of 33.2% for 1998 and 39.2% for 1999. 
Providing adequate user support ranked second in the 1999 survey report according to 
28.2% of the survey respondents, an increase from 26.5% in 1998, and 25% in 1997. 

An analysis of the 1994-1999 survey reports also shows that the use of technology 
in instruction has been rapidly rising. Specifically, data reported for the use of e-mail, 
Internet resources, and Web pages were examined over the six-year span. Table 1 
summarizes the findings. 



Table 1 

Summary Table of Technology Use in Higher Education 
(in Percentages by Responding Institutions) 



Variable 


1994 


1995 


1996 


1997 


1998 


1999 


E-mail 


9 


20 


25 


32 


44 


54 


Internet Resources 


NA 


11 


16 


25 


33 


39 


Class Web pages 


NA 


7 


9 


15 


23 


28 



(Adopted from Campus Computing Survey Reports 1994-1999) 
Note: Percentages reported contain rounding off errors 



E-mail had already been in use in 1994 rising to 54% by 1999 up from 44% in 
1998, and 20% in 1995. Similarly, use of Internet resources was 1 1% in 1995 and rose to 
39% in 1999. Similarly, the percentages of college courses using the WWW in the syllabus 
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rose from 7% in 1995 to 28% in 1999 compared to 23% in 1998 and 9% in 1996 (CCP 
1996). The most relevant finding to this study was the growth of the use of Web pages for 
classroom teaching. This number quadrupled by 1999 meaning that more than one in every 
four classes has a Web page. In fact, compared to e-mail and general use of Internet 
resources, use of Web pages has shown the highest rate of increase over the six-year 
period. 

In summary, over three consecutive annual surveys, providing assistance to faculty 
efforts to integrate technology into instruction, emerged as the most important issue 
confronting U.S. higher education in the past three years. Dr. Kenneth Green, project 
director, predicted that this will continue for the next two to three years (CCP, 1999). He 
also pointed out that for many institutions user support and instructional integration are 
complementary components of the broad challenge that involves the effective use of new 
technologies in teaching (CCP, 1999, p. 1). Interestingly, however. Green noted that the 
key information technology challenges in higher education involve people, not products, 
unlike in the past, when higher education was more concerned with building the 
infrastructure to support the technology. Green also observed that “what’s ahead for most 
faculty and most students is some kind of hybrid learning experience in which technology 
supplements, not supplants, both the content and the discourse that have been part of the 
traditional experience of going to college” (CCP, 1999, p.1-2). Given the above research 
findings, the following section presents what some scholars envision as lying ahead for 
integration of the WWW in classroom teaching. 
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2.3 The World Wide Web (WWW/Web) 

As stated earlier, the WWW is one of several services on the Internet. One 
definition of the Internet is a “...collection of networks that includes local area networks 
(LANs), dedicated computer lines, telephone lines, and satellite links” (Starr, 1997). The 
WWW is a world wide electronic medium for the storage, retrieval, and dissemination of 
linked information on the Internet. Moreover, the WWW is a graphical information 
provider that is geographically, time and platform independent (December & Randall, 
1996). 

2.3.1 History and Characteristics 

Chronologically, the World Wide Web was released onto the Internet in 1991 
(Zakon, 1996). By 1994, the World Wide Web was already becoming the most popular 
service on the Internet and with the highest traffic by 1996 (Zakon, 1996). Tim Berners- 
Lee, a physicist at Conseil Europeen pour la Recherche Nucleaire (CERN), proposed a 
hypertext project that later in December 1990 became a program known as the 
WorldWideWeb (Starr, 1997). The LineMode Browser was a tool developed to access Web 
data (Starr, 1997). A graphical user interface (GUI) was developed at the National Center 
for Supercomputing Applications (NCSA) and, in February 1993, became the first Web 
browser known as Mosaic (Starr, 1997). Currently, Netscape Communicator and Microsoft 
Explorer are the two browsers commonly known on the market. 

The WWW consists of a user interface that is consistent across many computers in 
addition to a set of standards that enable the interface to access a variety of document types 
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and information protocols (December & Randall, 1996). Text that is hyperlinked is known 
as hypertext. Hypertext MarkUp Language (HTML) is the common language used to code 
information on the Web. Hypertext is non-sequential text used to store information in a 
network of hyperlinked nodes (Seyer, 1991). Hyperlinking occurs when text is highlighted 
or underlined thereby marking an attachment to another piece of information. Hypermedia 
extends hypertext in two ways: it incorporates multimedia into hypertext documents and 
allows for graphic, audio and video elements to become links to other documents or 
multimedia elements (December & Randall, 1996). 

In summary, the WWW is perceived by many as the software choice on the 
Internet (Grau & Bartasis, 1995). Proponents of use of this medium highlight its 
hypermedia capabilities as one of its unique features in addition to its integration of various 
multimedia from video clips to three-dimensional images, claiming that the WWW is 
emerging as possibly the number one delivery medium for creative pedagogical use 
(Nichols, 1995). The WWW, like several other resources, provides information that can 
be used for teaching and learning. However, the quality, accuracy, and credibility of this 
information can not be taken for granted. The WWW differs from other resources in that it 
is has powerful features that allow users world-wide access to more abundant information 
on various topics through electronic databases created by people world wide and linked 
through hypertext, images, sound, and animation (December & Randall, 1996). 

The impact of the World Wide Web on education is evident in the changing student 
and teacher perceptions of space and time (Crossman, 1997). According to Crossman, this 
change marks a new threshold in how the WWW will be used as an instructional 
technology resource. Historically, technologies that tend to flourish are those that are 
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simpler to use and have a consistent standardized format, such is the case with the WWW 
in that it meets the first criterion of ease of use. Similarly, the use of the hypertext transfer 
protocol {http) also meets the second criterion of consistency by virtue of being the 
standard form of the uniform resource locator throughout the world. Previous complex 
procedures have been by-passed and there is continuous development of search engines 
using keyword searches. In addition to the above. Crossman (1997) pointed out the unique 
integrated attributes of the WWW, including text, graphics, photography, full-motion color 
video, and high fidelity sound make it the fastest growing communication system in 
history. According to Romiszowski (1997) 

It [the WWW] is the latest embodiment of hypertext/hypermedia environments, 
allowing practical implementation and use of hypertext environments to graduate 
from the relatively small stand-alone systems previously developed with tools such 
as HyperCard or ToolBook, to much larger and universally available systems of 
structured information, (p.27) 

2.3.2 The WWW as a Distributed Learning Environment 

When a given technology fosters a communication system or environment that is 
independent of space, time, and location, it has the capabilities of supporting a distributed 
learning environment. The WWW has such capabilities in that it allows access to 
knowledge and information databases independent of time, distance, and location 
constraints. In this distributed learning context, “the WWW can be used in combination 
with traditional classroom-based courses, with traditional distance learning courses, or can 
be used to create wholly virtual classrooms” (Saltzberg & Polyson, 1997). 
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Furthermore, since the WWW, by its very nature, distributes resources and 
information, it can be used in combination with electronic mail, Telnet, Usenet 
Newsgroups, video-conferencing and various other Internet tools and thereby transformed 
into a powerful interactive environment. Through the use of this environment and such 
capabilities, it is claimed that the WWW becomes the most ideal medium of delivery for a 
distributed learning model (Saltzberg & Polyson, 1997). Given this claim, a closer 
examination of current efforts to use the WWW for teaching and learning purposes follows. 

2.4 Web-Based Instruction (WBI) 

2.4.1 A Definition 

A review of current literature revealed several overlapping definitions of Web- 
based instruction (WBI). Currently, several references exist regarding use of the WWW in 
an educational context. Web-based learning (Hackbarth, 1997), Web-based instruction 
(Khan, 1997), Web-based training (Driscoll, 1997) and distributed-learning (Dede, 1995) 
are among the most frequently used terms. Barron (1998) identified several other 
classifications including Web-enhanced instruction; Web-managed instruction; and Web- 
delivered instruction. Barron pointed out that none of these arbitrary categories are 
exclusive (1998). 

A more specific definition is that proposed by Gillani and Relan (1997) whereby 
Web-based instruction “is interpreted broadly as any form of instructional delivery in 
which the World Wide Web is included as a tool” (p. 41). This definition seeks to 
differentiate Web-based instruction from traditional instruction in that the latter is broadly 
defined as “an instructional environment which, among other characteristics, occurs within 
confined walls and is primarily face-to face” (Gillani & Relan, 1997, 
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However, Khan (1997) provided an even more encompassing definition of Web-based 
instruction, which will be used for the purpose of this study. 

Web-based instruction (WBI) is a hypermedia-based instructional program 
which utilizes the attributes and resources of the World Wide Web to create a 
meaningful learning environment where learning is fostered and supported [italics 
added], (p. 6) 

Moreover, it is important to note that the overlap and inconsistencies, throughout 
research documents, in the use of terminology to describe Web-based instruction extends to 
the inconsistent use of terms to define the characteristics and attributes of this medium 
beginning with the use of the terms Web and WWW! 

For example, Bannan-Ritland, Harvey, and Milheim (1998) used the term elements 
to describe an instructional categorization scheme for effective design and delivery of 
instruction. While the authors did not define their use of the term elements they did use it 
to define course-related logistical elements such as a syllabus, schedule, and class notes and 
assignments. The authors also used the term components for instructor-selected Web links, 
Web-converted presentation slides, and Web-based tutorials. In addition, the authors used 
the term activities for posting of course-related information, organization of Web resources 
and links, interactive on-line communication, instructional modules including presentation 
of information and feedback related to specific content. El-Tigi and Branch (1997) used 
the term attributes to describe Web frames, tables, and image maps which are organized 
hyperlinked graphical displays of information afforded by the Web. Collis (1998) used the 
terms functions and functionalities for activities such as posting information on the Web; 
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making a calendar available on the WWW; providing feedback; extending lecture by 
increasing various forms of communication; and test-taking and scoring. 

Khan (1997), in his book entitled Web-based Instruction dedicated his introductory 
chapter to definitions as an attempt to classify and categorize the terminology. First he 
defined and explained the phenomenon of WBI by providing two distinct classifications a) 
components and b) features. According to Khan (1997) components are integral parts of 
the World Wide Web. Features are characteristics that are contributed by these 
components. Examples of the various components of WBI include: content development 
(e.g., instructional design and learning theories), multimedia (e.g., graphics, text, video and 
audio streaming), Internet tools (e.g., asynchronous such as e-mail, listservs, and 
newsgroups and synchronous such as Chat, IRC, MUDS), and hardware and software (e.g., 
computers, browsers, authoring programs, connection and service providers, and servers). 

Web-based features are represented as attributes that are conducive to learning and 
instruction. Key features include: the interactive nature of the medium, its global 
accessibility, distributed open system, and on-line searching. Additional features include: 
convenience, self-contained; non-discriminatory; cost-effective, environmentally friendly, 
and collaborative learning. In addition, Khan uses the term “components” to include a 
wide range of integral parts such as content development, multimedia, and tools. 

Second, Khan (1997) used the term elements when referring to entities of a 
component. Within the multimedia component for example, are smaller elements such as 
color, graphics, pictures, animations, text, navigational buttons, sound, music, and 
italicized, bolded, highlighted, and blinking text (personal communication, February 14^’’, 
2000). Khan’s classification is one way of synthesizing the various characteristics of the 
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World Wide Web when used in an academic setting. It is by no means an exhaustive 
classification and may require further modification as the WWW continues to evolve and 
WBI reaches maturity. However, it is one of the few, if not the only, attempt to define 
certain terminology emerging in the literature related to Web-based instruction. In sum, it is 
important to note that the literature, like in the overlap of the definition of WBI, also has 
not reached a consistent use of terminology to describe the above. 

In addition to Khan's (1997) definitions and usage of the terms described above, in 
this report, the term functions is used to describe ways in which the WWW can be used for 
teaching and learning purposes. These functions encompass user actions such as searching 
the WWW to access reference materials; downloading/printing information fi'om the course 
Web site; viewing a multi-media presentation/lecture; and communicating with the course 
instructor/ Teaching assistant(s). Furthermore, while Khan (1997) did not define those two 
terms, this author uses the term support in reference to instructional design efforts that 
contribute to the integration of WWW technology to create useful learning environments 
and include various instructional functions such as interaction, presentation, practice, 
guidance, elicitation, feedback, and assessment. Foster refers to the provision of various 
learning opportunities that provide remediation and enrichment to students' learning 
experiences. 

2.4.2 Typology of Course Web sites - WWW Technology Integration 

Currently, there are hundreds of educational Web sites. In higher education, 
numerous Web sites are developed to complement the curricula. There are hundreds of 
examples of faculty delivering their instructional materials on the WWW, the World 
Lecture Hall provides numerous course Web sites in over seventy five academic areas and 
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is available at http://wwwhost.cc.utexas.edu/world/instruction/index.html . The WWWDEV 
Members' Course List available at http://www.unb.ca/web/wwwdev/c3.html is a second 
example of directories listing courses from all over the United States. At a university level, 
the Syracuse University Faculty Syllabus Web site available at http://syllabus.svr.edu is an 
example of a growing list of campus-based course Web sites. Classification of these 
creations takes a variety of forms. 

Generally, course Web sites such as the above listings are offered in varying 
contexts to a diverse student population. A review of the literature resulted in finding very 
few attempts to provide a typology of use of the WWW for educational purposes. For 
example. The 'Educational Technology Special Issue dedicated to Web-based learning 
(Hackbarth, 1997) had only one article (out of nine articles) that attempted to classify these 
emerging Web sites. Quinlan (1997) described some of these educational Web sites as 
student-generated projects while others are subject-oriented syllabi of course material. 
Similarly, in Web-based Instruction (Khan, 1997), besides the editor, only Butler, provided 
a conceptual framework for categorizing uses of the WWW in the classroom (1997). 

Butler identified three ways of using the WWW in the classroom: (a) as a tool for 

bringing the world to the classroom, (b) as a tool for supporting classroom activities, and 
(c) as a tool for opening the classroom to the world. 

In 1998, the Paul Allen Foundation sponsored a virtual education contest for the 
most outstanding online course. A total of 182 entries were received from 148 institutions. 
These entries were judged by a panel of experts in educational technology (Kearsley, 

1998). Some salient characteristics of these courses were described by one of the expert 
panelists. Dr. Greg Kearsley in an article reporting the criteria used to identify the most 
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outstanding online courses. Several categories were identified to describe the most salient 
characteristics. These categories were more detailed and inclusive than any other previous 
attempts that the author is aware of. These categories are further described below; 

The salient characteristics of the 1 82 course Web sites were primarily categorized 
into: (a) course components, (b) relationship to onsite instruction, (c) pedagogy, (d) course 
development, and (e) student assessment. A wide range of what constitutes course Web 
sites were described by Kearsley (1998) as course components follows: 

Courses involved various combinations of these components, ranging from 
an online syllabus and use of mailing lists to full-fledged sites with lectures using 
streaming audio/video, chat rooms, conferences, and extensive quizzes or problem 
sets. Online course materials ranged from static documents (perhaps illustrated with 
graphics or photos) to highly interactive by virtue of links to other Web sites, 
animations, simulations, or exercises with feedback. Archives of student work could 
be completed assignments (often published as Web pages) or responses to questions 
from online conferences. It should be noted that successive versions of online 
courses tend to include more interactive components as instructors become more 
comfortable with online teaching and Internet capabilities, (p. 2) 

Relationship to on-site instruction varied among the courses from those that were 
supplements for traditional on-site classes to those that were totally online with no on-site 
regular meetings. The totally online courses included online final exams. Pedagogy 
reflected in the course entries ranged from traditional didactic with lecturing as the primary 
method of instruction, to novel discovery and problem-based learning methods. Overall, 
there was emphasis on student participation and group interaction thereby employing active 
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learning techniques. Other than this emphasis, the instructional strategies and methods 
employed were reported to be similar to traditional course design (Kearsley, 1998). 

Course development ranged from those created solely by a single instructor on 
his/her own initiative, to those developed by large teams with institutional resources and 
support. Kearsley (1998) further elaborated: 

Most courses appear to be developed by a small team of two or three people, 
generally an instructor with help from a colleague, designer/programmer, or student 
assistant. Some courses and web sites were created from scratch using HTML 
editors whereas others were developed in the context of web authoring systems and 
services such as TopClass, Lotus Notes/LeamingSpace or Real Education. Courses 
that involved chat or forums typically used commercially available packages such 
as WebCT, WebBoard, or NetMeeting. For multimedia materials, Shockwave, 
Acrobat, and Real Audio/Video were popular. One interesting difference across 
courses was the extent to which the personality of the instructor was present. 
Courses developed by single individuals tended to be quite distinctive whereas 
those created by teams using authoring systems were often fairly impersonal in 
nature, (p. 3) 

Student assessment was reported to be present in a variety of ways. Among the 
most common method reported was to have students respond to assignments, exercises or 
exams via e-mail. Kearsley (1998) fiirther elaborates: 

Many courses used online forms for quizzes or tests, which were 
automatically scored when completed, with immediate results displayed to the 
student. Most courses had some type of weekly assignments and major projects. 
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There was a lot of emphasis on group work in completing assignments and 
extensive use of peer evaluation schemes. The latter could be as simple as having 
students comment on each other's responses in a forum to a complex rating system. 
A number of courses required students to keep online journals or create portfolios 
for evaluation of their progress, (p. 3-4) 

For the purposes of this study, a preliminary analysis of approximately 300 
Syracuse University Web sites was conducted during the fall 1997 and the spring 1997 
semesters which generated a very basic typology of course Web sites. It is important to 
note that not all course Web sites available at Syracuse University were included in the 
analysis. Only those that were residing on the faculty server were examined 
(http://syllabus.syr.edu) because they were the study’s sampling frame (see section 3. 4.2.2 
for sampling procedures). The analysis revealed that those class Web sites were typically 
used to supplement class lectures with pointers to references and significant sites and/or as 
a communication tool between faculty and students. In some cases, use of a class Web site 
is a requirement, while in other cases it is optional. In terms of purpose and use, these Web 
sites have a wide range. Some Web sites may serve as instructional support resources for 
already existing traditional class sessions with functions similar to the typology described 
by Kearsley (1998). Other Web sites may serve more as bulletin boards or organizers to 
display and monitor various administrative aspects of the courses such as the course 
syllabi, class schedules, and housekeeping aimouncements. In this study, an instrument 
was developed to classify course Web sites according to their instructional quality. A 
rating scale was used to describe the functions and characteristics of each Web site (see 




section 3.5.1). 
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2.5 Highlights of WBI Research 

Reviews of decades of research on computer applications in education have 
reported that: computer-based learning approaches have improved student achievement, 
saved students and teachers time, and improved students’ attitude toward school and 
particular subjects (Thompson, Simonson, & Hargrave, 1996). The most prevalent studies 
of media use in education have been media comparison studies (Thompson, et. al., 1996). 

In their review of the research, Thompson et. al. (1996) identified several shortcomings of 
media comparison studies between two media or one media and traditional instruction. 

One major shortcoming was the tendency to focus on media as the independent variable 
and learning achievement as the dependent variable where the outcome would be to prove 
that a particular medium had greater effect on learning. Unfortunately, much of this type of 
research has been criticized because of design problems (Thompson, et. al., 1996) where 
instructional strategies, learner characteristics, and instructional content were confounding 
uncontrolled variables. 

Of importance to this study is Salmon and Clark's (1977) invalidation of the 
"media affects learning" argument that had been the driving cause of sixty years of media 
research. The 80s and early 90s have witnessed a major debate in the field on media 
research. This debate concluded that past decades have confounded media and method and 
that there is no benefit in studying media comparison studies that focus on differences in 
achievement (Clark, 1983, 1985, 1994a, 1994b; Kozma, 1991, 1994a, 1994b). A second 
major shortcoming of these studies is the focus on the significant versus non-significant 
differences on achievement between the groups (Clark & Surgrue, 1988, as cited in 
Thompson, et. al., 1996). The most outstanding result was that different media did not 
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significantly differ for achievement between groups (Clark & Surgue, 1988). In summary, 
a discussion of relevant literature for this chapter excludes all media comparison studies 
and focuses on (a) design and development literature on Web-based instruction and (b) 
empirical studies on student- perceptions of Web-based instructional environments. This 
review summarizes some current trends in media research specific to WWW technology. 

2.5.1 Design and Development Literature on WBI 

A review of current educational technology literature specific in scope to Web- 
based instruction resulted in a synthesis of several perspectives, recommendations, and 
some empirically-based findings for optimal instructional use of WWW technology. The 
overall perspective favors embracing the new technology, describing the WWW as a highly 
sophisticated, versatile communication channel for learning and teaching through its 
integrated multi-media features (Crossman, 1997; Duchastel, 1997; Ehrmann, 1997; 

Ritchie & Hoffman, 1996; Khan, 1997; Romiszowski, 1997; Starr, 1997). According to 
Reeves and Reeves (1997) many in this group predict that the WWW will revolutionize 
instruction and dramatically improve the effectiveness of education and training. There are 
also those who advocate course re-design for university instruction and pedagogical re- 
engineering using Web-based course support functionalities (Collis, 1998). Others 
highlight the promise of the WWW as a distributed learning environment (Dede, 1995; 
Oblinger & Maruyama, 1996; Saltzberg & Polyson, 1997), while there are some who are 
perhaps cautious if not skeptical (Descy, 1997; Reeves and Reeves, 1997). 

A closer examination reveals researchers’ positions that are generally in favor of the 
technology combined with overall high expectations for the WWW to fulfill its acclaimed 
promises. The journal oi Educational Technology (Hackbarth, 1997) featured a special 
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issue on Web-based learning. The overall tone of the contributing authors was quite 
positive and optimistic with regard to the rationale for using this technology. Several 
articles focused on ways to optimize use of the WWW for teaching and learning. Starr 
(1997), for example, observed that standard instructional design principles apply to the 
design of Web-based learning programs, but that the unique features of the WWW afford 
both opportunities and challenges. Quinlan (1997) employed a systematic design approach 
to utilization of the medium but also cautioned about its disorganized nature and its highly 
distracting propensities. El-Tigi and Branch (1997) focused on utilization of specific 
capabilities and attributes of the medium that may serve to heighten learners' interaction, 
control, and feedback as part of exercising their cognitive learning strategies. 

However, in spite of the above-mentioned efforts, this issue was critiqued for 
“being devoted entirely to technical issues (e.g. Starr, 1997) and teachers' perspectives 
(e.g., Berge, 1997)” (Kara & Kling, 2000, p.6) and perhaps not equally attending to 
student-centered research. Added to that, this issue did not provide any empirical findings 
on WBI. Further analysis of articles in this issue also revealed that the overall focus was 
on constructing design guidelines for developing Web-based instruction. Many of the 
authors proposed several strategies for optimizing use of the WWW including some 
prescriptive applications of their propositions. The shortcomings of the articles are in the 
lack of reporting actual empirical research findings to support their recommendations and 
claims. Nonetheless, it is important to take into consideration that this issue was published 
in 1997, at a time when there was a wide gap in educational technology literature on 
designing Web-based instruction. Currently, there is a proliferation of design guidelines 
and recommendations as can be seen in the works of Bonk and Cummings (1998); Collis 
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(1998); Khan (in press) Langenbach and Freimut (1997) but there is still a shortage in 
empirical findings to support the emerging theoretical positions. 

During the same year of 1997, a book edited by Badrul Khan was published by 
Educational Technology Publications. This is the first book in the educational technology 
field to be published on Web-based instruction and is one of the most widely cited 
references on WBI. The book includes various reports on the definitions, design, 
development, evaluation, and delivery of Web-based instruction. In sum, a closer 
examination of the above leads one to conclude that educational technologists have not yet 
provided higher education with tried and tested applied solutions. Most of the works cited 
above are not grounded in empirical-research. Kara and Kling (2000) explained this 
phenomenon as due to the "WWW being relatively young and is still in a testing stage, not 
an evaluation stage" (p.4). 

Indeed, the above observations and conclusions are not surprising as Merrill et al. 
(1996) observed “many persons associated with educational technology today are engaged 
in a flight from science” (p. 2) and where the structure of educational technology is 
becoming more relativistic that scientific. Specifically, several scholars from various 
disciplines criticized educational technology research pertaining to Web-based instruction 
for the lack of empirical research. Reeves and Reeves (1997), for example, pointed out that 
there exists little research evidence that supports claims for the effectiveness of Web-based 
instruction. On a larger scale, Owston (1997), observed that, while there is rapid growth of 
WWW use in traditional higher education classes, there is little research on its value and its 
impact in actually facilitating student learning (Owston, 1997). Some were even concerned 
about identifying empirical evidence to support recommendations on how the mainstream 
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educator can benefit from the innovative technology (Butler, 1997). Windschitl (1998) 
noted that research on the use of the WWW lacks disciplined scholarly articles where most 
of the published work is either anecdotal descriptions of technology implementation in 
classrooms or tends to be intuitive analysis of what works and what doesn't work with 
students (p.l). McClure & Lopata (1996) summed up the state of affairs in higher 
education as such; 

Despite the fact that many institutions of higher education have built 
significant networks and have cormected to the Internet as part of the evolving 
national Information Infrastructure (Nil), there is little knowledge of how such 
connectivity has affected the academic institution, in particular how has access to 
use of these resources affected teaching and learning and other aspects of academic 
life (p. 1). 

In spite of the above observations, empirical research related to integration of 
WWW technology into instruction is gaining some ground. Since 1996, several studies 
were published with the purpose of providing empirical research on the effectiveness of 
Web-based instruction. These studies are most relevant to this study because they are 
evaluative in purpose and are student-perception studies. The following section highlights 
some examples conducted in the past five years. 

2.5.2 Empirical Studies on Student-Perceptions of WBI Environments 

Several studies focusing on student-centered empirical research were reviewed 
(Andersen & Joerg, 1996; Bonk & Cummings, 1998; Collis, 1998; Kara & Kling, 2000; 
Hudson, Greer, Buhler, and Paugh, 1998; Jiang, 1998; Reinhart, 1999; Sarapuu & Adojaan, 
1998; Schlough & Bhuripanyo, 1998; Van Rennes & Collis, 1998). The participants in 
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these studies were primarily adults, some were campus students, while others were distance 
learners. The participants were undergraduate students (Bonk & Cummings, 1998; 
Reinhart, 1999); graduate students (Collis, 1998; Hara & Kling, 2000); high school 
teachers from Estonia (Sarapuu & Adojaan, 1998); and adult distance learners (Hudson, 
Greer, Buhler, & Paugh, 1998; Jiang, 1998). The context in which these studies were 
conducted varied ranging from distance education courses for part-time adults (Hara & 
Kling, 2000; Hudson, et al., 1998; Jiang, 1998) to campus-based courses using the WWW 
as support (Andersen & Joerg, 1996) or Web-based courses offered to campus-based 
students (Bonk & Cummings, 1998). 

2.5.2. 1 Research Questions 

These studies are among the few that investigated fundamental research questions 
related to WBI posited in the literature. The questions varied in focus and domain of 
inquiry such as those concerning rate of use and adoption posited by Anderson and Joerg 
(1996) were: is the WWW perceived by students and instructors as a valuable aid to the 
teaching/leaming process? and what are the barriers (as perceived by both teachers and 
students) to adoption and use of this technology? (1996). Others were concerned with 
affective issues of students’ learning experiences. For example, Hara & Kling (2000) 
posited affective questions such as: how do students’ frustrations in Web-based distance 
education courses inhibit their educational opportunities? and how do students deal with 
those frustration? Hudson et. al. (1998) were concerned with students’ perceptions of 
factors related to administrative support, technology use, and organization of a Web-based 
distance course. Similarly, Bonk and Cummings (1998) were concerned with students’ 
success and struggles of Web-based learning. Other types of questions were more 
validation of the course Web sites, from a student perspective, in terms of content (Sarapuu 
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& Adojaan, 1998), user interface design (van Rennes & Collis, 1998), and course design 
and development (Bonk & Cummings, 1998). Other studies were more specific to learning 
by examining factors influencing students' perceptions of online learning (Jiang, 1998), and 
student motivation, self-efficacy and task difficulty in Web-based instruction (Reinhart, 
1999). 

2. 5.2. 2 Methodology 

These studies varied in their data collection methods. Some employed a case-based 
study method (Kara & Kling, 2000). The majority employed online or print-based survey 
method (Andersen & Joerg, 1996; van Rennes & Collis, 1998; Reinhart, 1999; Sarapuu & 
Adojaan, 1998). Others employed observations, document reviews, content analysis of 
online correspondence, and interviews (Bonk & Cummings 1998; Kara & Kling, 2000; 
Jiang, 1998). 

2. 5.2. 3 Relevant Findings 

Findings of the studies were similar in that they were student-centered but different 
in the domain of inquiry pertaining to students’ learning experiences as demonstrated in 
each study research questions stated above. What is also common to all these studies is 
that they engaged in formative evaluation methods used in instructional design to pilot and 
field test instructional interventions (Dick & Carey, 1996) specific to a Web-based context. 
Findings from these studies can be classified into two categories: (a) student-perception of 
learning studies and (b) student-evaluation of Web site development studies. In some 
cases, the study was a mix of both. The following section summarizes the pertinent 
findings of each study. 
a) Student-Perception of Learning Studies 
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Anderson and Joerg (1996) found that both students and instructors perceived the 
WWW as a valued education enhancement. Significant barriers to adoption including 
access restrictions, questions related to relative advantages of the technology, and problems 
in the creation and organization of large quantities of WWW pages.. Students reported that 
the WWW changed the dynamics of access to class materials at any time from different 
locations. They also reported several enhancements in the speed with which class materials 
were revised by instructors; in their capacity to review materials, and in the reduction of 
paper consumption. Students appreciated the hypertext linkages embedded in the course 
material. Negative comments were related to access problems due to restrictions in 
hardware and telephone lines, and the inability to personally annotate class notes and 
materials. Students also expressed concerns over the time spent on external Web sites. 
They pointed out that it was "time-wasting". 

Bonk and Cummings (1998) found that students appreciated the openness of Web- 
based conferencing as well as the collaborative spirit of peers in providing feedback and 
queries. Students reported that they felt less inhibited when using conferencing tools which 
enabled them to engage in controversial discussions. Students appreciated having some 
personal choice and alternatives within class assignments. The researchers also found that 
electronic syllabi must be clearer on the Web than in traditional classrooms to avoid 
student confusion and anger in a Web-based environment where face-to-face immediate 
clarifications were missing. 

Kara and Kling (2000) found students' fhistrations with a Web-based distance 
education course originated from two sources: technological problems and pedagogical 
issues. Technological problems included students' difficulty in obtaining technical support 
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especially for students with inadequate computer skills. Pedagogically frustrating issues 
included minimal and not timely feedback from the instructor and ambiguous instructions 
on the Web or through e-mail. The impact of these frustrating experiences resulted in some 
students dropping out of the course, negative attitude toward Web-based distance 
education, and "giving up" on the formal course content. 

Hudson et. al. (1998) examined two relevant aspects: support needed by students 
and essential organization of the course. They found that in terms of support for Web- 
based training courses, trainees appreciated support from each other and from family 
members as they perceived Web courses to be time-consuming. Novice students 
appreciated the support from peers especially in overcoming technological hurdles such as 
sending an assignment, receiving instructions on sending e-mail attachments, posting 
comments on the bulletin board, etc. In terms of course organization, findings included 
posting tips to follow when using various course functions; minimizing changes during the 
course; and posting due dates well ahead of time as not every student is online daily or 
even every other day. 

Jiang (1998) found that socio-collaborative course environments are more 
conducive to perceived learning. Second, grades for discussion and requirements for 
discussion were significantly and positively related to students' perceived learning. Third, 
a course with a balance between BIG (beyond information given) and WIG (without the 
information given) is more conducive to perceived learning. Results showed that a certain 
amount of instructional support (content presentation and responses) is needed. In 
addition, a relationship between structure and nature of instructors' questions and patterns 
of students' responses was found. Courses with high perceptions of student learning were 
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elaborately designed, focused on students' own experiences and sought multiple 
perspectives. Likewise students' responses in these courses were drawn from personal 
experiences, familiar examples, and went beyond the understanding level. 

Reinhart (1999) found a positive relationship between students' self-efficacy for 
WBI and motivation to learn from WBI. Second, students' motivation to learn from WBI 
was positively related to achievement with WBI. Third, students' control of learning may 
not be an important issue for certain types of WBI activities such as WebQuests because 
students' motivation and achievement levels did not appear to be significantly related to 
their 'control of learning beliefs'. 
b) Student-Evaluation of Web site Development Studies 

Sarapuu and Adojaan (1998) developed an evaluation scale of educational Web 

sites and reported results based on their participants' use of the evaluation scale. The scale 
focused on technical, pedagogical, and subject-related criteria of evaluation. Unfortunately 
the study reported only comparative results on evaluations of two courses on each of the 
above stated dimensions. While the instrument does show consistent variation between the 
two sites examined, no information is provided on the validity of the instrument or on any 
theoretical basis for the development of the items in the instrument. 

Schlough and Bhuripanyo (1998) found that a Web-based course using Asymetrix 
Toolbook II Instructor software was a viable way to deliver teacher education instruction to 
distant learners in remote locations. Students’ evaluation of the course indicated that the 
strengths of Web-based delivery was in the convenience in terms of time and space; the 
promotion of individual self-paced learning; the open display of course products; the clarity 
and importance of online content in relation to course objectives; and the increased sense of 
connection with the course instructor. Weaknesses in the Web-based delivery system were 
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in the requirement for self-discipline; that it was not appropriate for all learning styles; and 
that it had the potential for being impersonal as not all students worked together. 

In their study of student evaluations of a user-based interface design for Web 
courses, van Rennes and Collis (1998) found that students preferred a "sequential time" as 
a navigational approach to course Web sites as opposed to a metaphor of "Centers" that 
was used by the instructors. A sequential time is a way of organizing the course Web sites 
by what is needed for a given week. Whereas the "Center" metaphor means that the course 
is organized by virtual locations of information such as the "Production Center", "the Study 
Center", and the "Search Center". 

In summary, these studies are pioneering efforts in providing empirical evidence for 
the effectiveness (or lack of) WBI in both distance learning and campus-based contexts. 
They shared a common goal in that they sought to gain a better understanding of the design 
and utilization of Web-based instruction. Replication of these findings is necessary. This 
study contributes to this growing body of empirical research on WBI. This study is similar 
to the above studies in that it investigated factors that affect students’ learning experiences 
from a student perspective and it also engaged in instructional course Web site evaluation 
from an instructional designer’s perspective and students' perspectives. The study makes 
additional contributions to the above literature in that it covers students’ perceptions of the 
usefulness of theory-based instructional functions of the WWW. Relevant findings from 
the above studies in association with this study are further presented and discussed in 
chapter six. 

In conclusion, research toward the identification of optimal functions provided 
through the capabilities of WWW that can be used to support instruction thus increasing its 
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utility and value to students and faculty is a necessary next step. These functions need to 
be identified, tested, and analyzed in an empirical setting. This means a study of these 
specific instructional functions that can be supported by Web technology. Furthermore, a 
review of the literature has shown that there has been no cross-sectional studies conducted 
campus-wide and across disciplines. This study contributes to closing this gap in the 
literature. Support for this endeavor was drawn from the principles of instructional design. 
The following section presents a summary of what is considered effective instruction and 
how theory becomes practice in a Web-based environment. 

2.6 Effective Instruction - Theory and Practice 

Instruction can be defined as a “purposeful interaction to increase a learner’s 
knowledge or skills in a specific, pre-determined fashion” (Ritchie & Hoffman, 1996). 
Effective instruction is "instruction that enables students to acquire specified skills, 
knowledge, and attitudes. Effective instruction is also instruction that students enjoy.” 
(Dick & Reiser, 1989 p. 2). According to Dick and Reiser, effective instruction can be 
measured by how students perform on a particular learning task and how they feel as a 
result of the instruction they receive (1989). Student learning is promoted through 
instructional design procedures and tools that are based on “correct” instructional strategies 
(Merrill, Drake, Lacy, Pratt, & the ID 2 Research Group, 1996). Instructional design 
facilitates effective learning through practical applications combining learning theory with 
instructional design models and principles (Gredler, 1992). Moreover, “there is a body of 
knowledge and skill that has been developed and archived by generations of scholars, 
scientists, technologists, artists, and others. The purpose of instructional design is to 
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develop the experiences and environments that facilitate [Italics added] the students’ 
acquisition of this knowledge and skill” (Merrill, et. al. 1996, p. 2). 
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The significance of utilizing the Web for instructional purposes lies in its power to 
deliver large amounts of information in an open system through the use of hyperlinking. 
However, there seems to be an implicit belief that “speedy access to lots of information 
equals learning” (Sherritt & Basom, 1997, p.8). It therefore becomes necessary to make 
the distinction between information and instruction. One distinction was aptly stated by 
Duchastel (1997) as follows: 

. . .it [That information is central to learning] does not imply that knowledge 
is but an accumulation of information; rather it means that it is information that 
generally triggers transformations of knowledge that bring growth and development 
of intellectual skill. These do not occur in a vacuum but rather build on concrete 
information that either questions current thinking or provides supportive feedback 
to help elaborate a growing viewpoint. How that information is obtained is of little 
consequence; what is important is that it is available at the time of need; and in 
principle, greater accessibility leads to actual availability when needed (p.6). 
Consequently, while the WWW promotes access to a wide-reaching range of 
resources, it does not, by design, facilitate learning (Hill & Haimafm, 1997) or foster and 
support it (Khan, 1997). Neither does publishing a WWW page with links to other pages 
or other sources, constitute instruction (Ritchie & Hoffman, 1996). Bannan-Ritland, 
Harvey, and Milheim (1998) described the current situation as follows: 

The mere use of the World Wide Web in a specific course does not 
automatically imply effective instructional activities or quality instruction for the 
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intended learners since many courses are simply a juxtaposition of a variety of 
instructional and non-instructional Web-based elements, with some courses 
utilizing the Web in a very limited manner and others using Web-based activities to 
provide a totally online instructional experience. In fact, while many instructional 
elements that exist in stand-alone Web pages or as parts of overall Web sites may 
serve valuable functions in supporting course interaction and/or providing 
administrative guidance, they may not, in and of themselves, significantly 
contribute to meaningful learning, (p. 77) 

Consequently, when designing an instructional Web site, certain principles of 
instructional design ought to be taken into consideration as a means of providing quality 
instruction and the support and fostering of meaningful learning that are essential 
components in the definition adopted for WBI. So how is this made possible? Many highly 
reputed instructional designers would agree that there are certain functions that ought to be 
present in an instructional sequence to meet the above criteria. These functions would 
motivate students by explaining what is to be learned; help students to recall previous 
knowledge; present instructional material in attractive and meaningful diverse ways; elicit 
performance; provide guidance and feedback; assess comprehension, application, 
synthesis; and provide enrichment and remediation to facilitate transfer (Bloom et al., 

1964; Dick & Reiser, 1996; Dick & Carey, 1996; Gagne, 1975; Gagne, Briggs, & Wager, 
1992; Grau & Bartasis, 1998; Merrill, 1980, Merrill et. al., 1996). Based on the above 
rationale, incorporating the above functions into instructional interventions designed to be 
delivered on the Web is one means of determining the instructional quality of a given Web 
site. This could be achieved by assessing the extent to which these events exist in the 
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design of the site. The work of Robert Gagne (1965, 1975) in designing effective 
instruction, also known as the nine events of instruction provides a theoretical framework 
that is widely known and used in the instructional design that specifically targets the micro 
level of lesson design. The following section further defines and explains the nine events 
of instruction and provides examples of application in research studies. 

2.6.1 Gagne’ s Nine Events of Instruction 

Instructional events are a set of communications that aid the learner in the learning 
process. Furthermore, good instruction works hand in hand with the internal learning process 
by providing the necessary stimulant or reinforcement desired (Gagne, Briggs, and Wager, 
1992). Effective instruction necessitates that the external conditions for learning support the 
internal conditions for learning. The nine events represent a structured, systematic way of 
breaking down the learning process into stages of learning (see Table 2). At each stage, there 
is a specific event that ought to occur to induce learning. Gagne, Wager, and Rojas (1981) 
provided guidelines to use of the nine events of instruction as a framework for planning and 
authoring computer-assisted instruction lessons. The events of instruction represent a 
sequence of steps that support and "stimulate the learner so as to make learning readily 
occur" (p.l8), the rationale being there are several steps to learning that could be 
accompanied by several stages to the series of displays that support and enhance learning. 
Gagne et al. (1981) stated that "in any complete act of learning, there must be nine events 
of instruction" (p.l8) that complement each act of learning. 

Table 2 outlines the nine events of instruction, provides a rationale and purpose for 
each event, and illustrates the connection between these events and the related learning 
principle. For example, gaining attention is an instructional event that serves to stimulate 
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reception of information. Its purpose is to direct learner’s activity toward an objective 
through motivational strategies. These strategies make reception of information more 
possible due to increased perception of the relevancy and meaningfulness of the 
information to learner’s needs. Similarly, enhancing retention and learning transfer is 
another instructional event that serves to facilitate transfer of newly acquired knowledge to 
new situations. The purpose of this event therefore is to enable the learner to retain 
information for useful purposes and future recall. It also provides opportunities for the 
creation of rich meaningful associations of isolated facts and generalizations of new 
information through its application to a range of practical situations. 

Furthermore, it is important to point out that not all nine events need always be 
present in one lesson. Yet, "the planning of Computer Assisted Instruction (CAI) needs to 
make potential provisions for the display of frames containing print and diagrams to reflect 
all of the nine events" (Gagne et al., p.l9). "The sequential order of these events is roughly 
from one to nine, although reversals within the sequence may sometimes be desirable but 
restricted to events that basically provide 'stimuli to be used in learning' such as events that 
stimulate recall, present new material, and guide learning. Furthermore, the decision to 
exclude an event must be deliberate (Gagne et al. p.l9). 
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A Matrix Showing the Relationship Between Gagne’s Nine Events of Instruction, Learning Principles, Rationale, and Related 
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Table 2 (Continued) 

A Matrix Showing the Relationship Between Gagne’s Nine Events of Instruction, Learning Principles, Rationale, and 
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A Matrix Showing the Relationship Between Gagne’s Nine Events of Instruction, Learning Principles, Rationale, 
and Related Studies in Computer Assisted Instruction (CAl) 
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(adapted and extended from Gagne', Wager, & Rojas , 1981) 



Since there are no research studies to date that report findings from having used the 
nine events of instruction to guide the design of instructional Web sites, evidence for the 
effectiveness of the nine events of instruction was found in research studies in the domain 
of Computer Assisted Instruction (CAI). Wang and Sleeman (1994) in their review of 
CAI research identified several studies that demonstrated the usefulness and effectiveness 
of the nine events when applied to CAI context. Research studies reviewed and cited by 
Wang and Sleeman (1994), in addition to other studies, are included in Table 2. 

However, it is important to emphasize that Gagne’s work is considered the legacy of the 
field of instructional design: theory and practice (Richey, 1996; Richey, 2000; Ritchie & 
Hoffman, 1996; Smith & Ragan, 1996). Gagne’s (1965, 1975) principles are generally 
considered to be exemplary of traditional instructional design principles and the most 
comprehensive and fundamental constituents of effective instructional design (Fields, 

1996; Richey, 1996; Richey, 2000, Smith and Tillman, 1996). 

2.6.2 Application of Gagne’s Nine Events of Instruction to a Web-based Context 

While Gagne’s work was primarily carried out in the 70s and 80s before the creation 
of the Internet, a review of the literature pertaining to current instructional design theories for 
WBI pointed to several advocates of the use of Gagne’s nine events of instruction to design 
Web-based courses. Ritchie and Hoffman (1996) provided suggestions on how each of the 
nine events could be applied to Web design. Grau and Bartasis (1998) advocated the use of 
the nine events of instruction to evaluate hypermedia programs to determine the extent to 
which they are instructional. The medium of instruction primarily acts as a vehicle but can be 
manipulated in ways that optimize delivery of instmction (Clark, 1983, 1994a; Kozma, 1994a). 
If there is no apparent instructional design blue-print, this information could become 
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overwhelmingly unstructured thereby impeding the learning process (El-Tigi & Branch, 
1997). Therefore, it becomes necessary to design instructional material utilizing 
instructional design principles to present instruction on the Web in ways that facilitate 
learning and optimize the multi-dimensionality of the Web and its non-linear capabilities. In 
other words to provide the necessary support and fostering of a meaningful learning 
environment essential to Web-based instruction as previously defined by Khan (1997). One 
way to do that would be to apply Gagne’s nine events of instruction to the design of Web- 
based courses. A major contribution of this study is to apply Gagne’s nine events of 
instruction to the evaluation of course Web sites from students’ perspectives on the quality 
and usefulness of the course Web sites as an instructional resource. 

Some limitations to be noted are that, while the events of instruction would most 
probably generate a more effectively designed instructional program, there are many other 
unexplored opportunities and constraints in practical use of the medium that may not 
necessarily be addressed by the nine events. The nine events are procedural steps for the 
design of instructional interventions and do not provide rich details on how to design 
meaningful learning environments where students constructively create knowledge that are 
also specific to a Web-based environment. Such a limitation of the model is not of particular 
relevance to this study however. Basically, the model provides basic ingredients to 
successful teaching and learning practices as external stimuli to support and foster internal 
cognitive processes of learning (see Table 2 for a match between learning theory and nine 
events). 

2.6.3 The WWW as Lever to Good Teaching and Learning Practices 
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A second part of the conceptual framework provided in chapter one (see Figure 
A) focuses on teaching and learning practices in a higher education context. It is 
adopted to ground the study in practical applications. The higher education efforts to 
integrate technology in teaching and learning have led to the rise of a project like the 
Annenberg/CPB Flashlight Planning Project which began in 1994 to help educators 
answer questions about technology and educational improvement (Ehrmann & Zuniga, 
1997). This project is “designed to help institution(s) learn whether technology is 
helping them improve practices and outcomes over time and, if not, why not” 
(Ehrmann, 1997a, p. 1) 

Members of this project emphasize that educators need to understand how their 
uses of technologies such as computing, video, and telecommunication are enabling 
improvement in their educational strategies. Moreover, educators also want to know 
whether such improved strategies promote desired improvement in what students have 
learned by the time they complete the program (Ehrmann, 1997a). The major concern 
is not necessarily measuring learning achievement as was the case in the past with any 
introduction of new technology, but more of the value-added that new technologies 
bring to the instructional environment. Stephen Ehrmann, director of the Flashlight 
Project - The American Association for Higher Education (AAHE) summed up the 
problem as follows: 

Many people assume that evaluation of the outcomes of technology 
investments is easy: "Just see whether students are learning more! But the real 
value-added from technology usually comes when 
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instructional objectives change, which means tests change, too. It's useless to 
discover that students scored 80% on one test and 90% (or 80% or 75%) on a 
different test administered three years ago. On the other hand, if you can discover 
that: 

• the conditions for good learning have improved (as measured by 
increased implementation of the seven principles); and 

• the faculty and the students believe that their use of technology was 
substantially helpful, then you've learned something important. 

Similarly it would be useful to discover that collaborative learning is down and 
that e-mail has been problematic. Or even that collaborative learning is extensive 
but e-mail is widely seen as alienating (Ehrmann, 1997a, p. 5). 

2.6.4 The Seven Principles of Good Teaching and Learning Practices 

The principles were created originally in 1 987 by Art Chickering and Zelda Gamson with support from 
Lilly Endowment, the Johnson Foundation and the American Association of Higher Education (AAHE) 
(Chickering & Ehrmann, 1996). According to the authors they are the result of distilled findings from 
decades of research on the undergraduate experience. The seven principles of good practice employing 
technology as a lever are described below: 

1. Good practice encourages contacts between students and faculty 

Electronic mail, computer conferencing, and the World Wide Web increase 
opportunities for students and faculty to interact and communicate much more efficiently 
than before. In addition, the nature of communication becomes more thoughtful, 
deliberate, convenient and safe. The negative aspects of face-to face confrontation are 
lessened. 
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2. Good practice develops reciprocity and cooperation among students 

Through the use of the above mentioned technologies, both synchronous and 
asynchronous communication with peers and experts in the field is increased. Student 
collaboration is encouraged. Through collaboration and better communication, skills such as 
problem-solving, leadership, and active learning are fostered. 

2. Good practice uses active learning techniques 

Active learning is an essential component of the learning process. Students learn through 
discussions, reflective writing, making meaning out of what they are learning by relating it to 
previous knowledge and skills gained. Technology can be used as a learning tool such as using 
the Web as a search tool for large databases of information. Through Java-based simulations, 
students can study simulations in physics where they can manipulate the data source and instantly 
see resulting changes. 

4. Good practice gives prompt feedback 

The use of e-mail to support person-to person(s) feedback is one way of providing 
feedback. The multimedia capabilities of the Web allow for an almost seamless integration of 
video clips, sound, images, and animation thereby allowing for multiple modes of communication 
through feedback and assessments. 

5. Good practice emphasizes time on task 

Time on task is a requirement for effective learning. Teaching strategies that make use of the 
Internet can be time-saving in many ways. By allowing students to learn at home or at work, they 
can save time on commuting to and from campus, finding parking places, or going to the library 
where they have to physically run around to retrieve resources, thus increasing productivity. 
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Using the Internet to communicate is also time efficient due to its synchronous and asynchronous 
nature. 

6. Good practice communicates high expectations 

Expecting students to perform well becomes a self-fulfilling prophecy (Chickering & 

Ehrmann, 1996). New technologies can communicate high expectations by providing access to all 
sorts of information that can be easily distributed between people worldwide. Samples of work 
can be shared as exemplary work or even for peer reviewing. Students are exposed to a multitude 
of perspectives, data sources, and real-life problems that provide them with powerful learning 
challenges that were not so readily available prior to the Internet. 

7. Good practice respects diverse talents and ways of learning 

Diversity in learning styles and perspectives enriches the mind. Technological resources can 
accommodate different learning styles and teaching methods. The multi-dimensionality and multi- 
modality of the World Wide Web accommodate and foster this diversity. 

The Flashlight Current Student Inventory (CSI) is a toolkit developed by the Flashlight Project 
for creating periodic studies of students’ perspectives of changes in teaching and learning practices 
using the seven principles of good teaching and learning practice in undergraduate education 
(Chickering & Gamson, 1987; Chickering & Ehrmann, 1996) as a theoretical underpinning. Five 
institutions helped design and test Flashlight: Indiana University Purdue University Indianapolis 
(lUPUI), Washington State University (WSU), Education Network of Maine, Maricopa 
Community College, and the Rochester Institute of Technology (Elumann & Zuniga, 1997). 
Several case studies have been reported using CSI to create student surveys to explore three 
different uses of technology in three different contexts at WSU (Brown, 1998) and at lUPUI 
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(Harrington, 1998). These studies, however were primarily pilot studies and the finding are not 
directly related to the focus of this study. ^ 

The author has sought the support of the Flashlight Project CSI in the design of 
the study survey instrument. Specifically, it was used to create a section in the 
questionnaire that indicates how much time students spent in a typical week using their 
course Web sites and also to create the open-ended student satisfaction questionnaire 
items. These items are identified in chapters three and four. 

2.7 Chapter Summary 

This chapter placed the present study in the larger context of educational research 
pertaining to use of the Internet, with a particular focus on the WWW and theories on 
instructional effectiveness. The chapter introduced the phenomenon of study, students' 
perceptions of the WWW, and provided empirical evidence for the WWW's growing 
presence in higher education. The history and characteristics of WWW technology were 
discussed. Web-based instruction was defined and a typology described. WBI highlights 
of empirical research were discussed. Two sets of pedagogical principles were presented 
and discussed in the context of designing WBI and as the basis for explaining the study 
outcomes. Details of the study context, research methodology, instrument development, 
and data collection procedures and analysis are described in the next chapter. 



^ The author has contacted Dr. Stephen Ehrmann on several occasions to identify any other studies that 
used the 7 principles but was unsuccessful in finding other than what is reported. 
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CHAPTER THREE 

Methodology 

3.1 Introduction 

This chapter presents and delineates the research methodology employed in the 
study. The chapter is divided into the following sections: the research questions, the 
context of the study, and the methodological approach which encompasses: choice of 
methods, sampling procedures, sample representativeness, and approach to increasing 
response rate. The remaining sections of the chapter briefly introduce the instruments 
and conclude with a summary of data coding, cleaning, and analysis procedures. 

3.2 Research Questions 

Included is a brief explanation of what each question serves to address in relation to the 
purposes of the study. 

1. How did students allocate their time using their course Web sites? 

2. What was the quality of the course Web sites according to students? 

3. How useful did students perceive their course Web sites? 

4. What demographics and characteristics influenced students’ use and perceptions 
of quality and usefulness of course Web sites? 

5. What were some student reported barriers to and facilitators of use of the course 
Web sites and overall impact of the WWW on students' learning experiences? 

Question #1 examines students' use of their course Web sites. It also provides data on 
specific time spent on each function, in a typical week, that the course Web site makes 





available. The focus is on identifying users and non-users of various Web site functions. 
The question also serves to provide information on the ranking of each function in terms 
of time spent. 

Question #2 examines the quality of course Web sites according to students. The 
focus is on describing specific Web site functions that were most and least appreciated by 
students. 

Question #3 examines students' perceptions of the usefulness of various functions. 
The focus is on describing specific Web site functions that students found most and least 
useful. 

Question #4 focuses on gaining insights into some contextually related demographics 
and characteristics that may influence students' perceptions on the various study 
dimensions. This question is answered by examining statistical relationships particularly 
with demographics and characteristics of the respondents. These demographic variables 
such as age, sex, academic class, and GPA have traditionally been explored in the social 
sciences and in educational media research (Thompson, Simpson, & Hargrave, 1996). 
However, there are other variables that have not been explored prior to this study in a 
context where the WWW is used as an instructional resource to campus-based teaching 
and learning. Variables of particular interest to the author are: student residential 
proximity to campus, regular access to computers, modem speed, type of connection, 
reasons for taking the course, and nationality of students. These variables were selected 
based on interviews conducted by the author with ten students enrolled in the spring of 
1997. The pilot course was one of the classes that made the final screening for course 
Web sites. Another reason is the author's familiarity with the context of the study. 
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Examination of relationships between demographics and student characteristics 
and the various dimensions of the study: Time Spent, Quality, and Usefulness provides 
empirical data for audience analysis. In a survey and analysis of instructional design and 
development models, Gustafson and Powell (1991) and Gustafson and Branch (1996) 
emphasized audience analysis as a necessary stage in instructional design systems. 
Similarly, Dick and Carey (1996) and Romiszowski (1984) discussed at great length the 
importance of audience analysis in designing instructional interventions which entails 
familiarization with student knowledge base and skills, access to computers, Internet 
skills, learning environment, and other more traditional demographics such as age, sex, 
GPA, and academic class. 

Question #5 focuses on gaining insights into some contextual factors related to the 
students' learning environment that might explain and guide interpretation of findings. 
This question is answered by identifying emergent themes from narrative data pertaining 
to barriers to and facilitators of the use of course Web sites from students' narratives on 
the open-ended sections of the questionnaire. Students' perceptions of the impact of the 
WWW on their learning experiences are also elicited. This question also taps into 
qualitative data that the pre-ordinately designed questions may have missed. 

3.3 The Context of the Study 

Syracuse University (SU) is a Research II private university under the Carnegie 
Classification (Directory of Higher Education, 2000). Located in central New York 
State, it was founded in 1870 as an independent coeducational institution (About SU, 
2000). Syracuse University promotes itself as working toward being a more student- 
centered research university that makes student learning its highest priority. 
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Demographically, at the time this study was conducted during the fall of 1998, 50 states, 

4 territories, and several countries (unspecified), were represented. A total of 12,130 
undergraduates, 6,163 graduate and law students, and 1,681 foreign residents comprised 
the student body on campus for fall 1998 (Institutional Facts and Figures, 2000). 

Syracuse University’s academic mission is to promote learning through research, 
scholarship, creative accomplishment, and service. Its vision is to be the leading 
student-centered research university. 

3.3.1 Access to Computers at Syracuse University 

Students may use any of the 700 DOSAVindows and Macintosh personal 
computers across campus without charge, regardless of their majors. Computers have 
word processing, Web browsers, graphics, statistical and data analysis, database 
management, programming, and communications capabilities such as telnet and ftp. 
Interconnected to the University's high-speed network, these computers provide access to 
a Unix timesharing network and the Internet (About SU, 2000). It is worth noting that 
one class from the State University of New York - Environmental School of Forestry 
(SUNY-ESF) was included in the study because its course Web site resided on the SU 
faculty server. When the study was conducted, students attending SUNY-ESF had access 
to most SU facilities including SU computer clusters. 
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3.3.2 Faculty Support to Use the WWW as an Instructional Resource * 

At Syracuse University, Web sites specifically designed for teaching and learning 
purposes are the responsibility of the individual faculty teaching the course. The faculty 
member either personally develops the Web site or a student is assigned by the course 
instructor to the development task. 

As of the fall 1998, there were over 400 Web sites on the Syracuse University 
centralized faculty server for Web-based material, some dating back as early as the fall 
1995. Some sites continue to be updated and modified for use each time the course is 
offered, while some others are abandoned for various reasons not relevant to this study. 

In terms of physical location and access, many of these sites actually reside on the 
publicly accessible faculty server housed at the Faculty Academic Computing Support 
Services (FACSS) computer labs, while many others reside either on departmental 
servers or on the Syracuse University server for personal home pages and are then linked 
to the faculty server. There are no explicit university policy that specifies where these 
Web sites may reside. The faculty Web server is a means of providing additional 
memory capacity for faculty to store their information and as a service to maintain the 
server offered by FACSS. However, the actual maintenance of the site is the 
responsibility of the faculty member himself/herself. The author's own experience in 
developing and maintaining a course Web site residing on the SU faculty server confirms 
the above report. 



Based on observations conducted on Web sites since the fall 1997 and several personal visits and phone conversations 
with key personnel in the Faculty Academic Computing Support Services (FACSS) at Syracuse University, the 
following is a description of the status of course Web sites during the academic year of 1997-1998. 
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3.3.3 Course Web Sites on the Faculty Server 

Since the fall of 1995, many different types of Web sites have been made available to 
Syracuse University students. They vary in type and purpose ranging from 
predominantly informational bulletin boards and an electronic version of course syllabi, 
to sites that engage students in various instructional activities. The latter were classified 
as closer to what has been defined in chapter two as instructional as opposed to 
informational (see section 2.6). Taking the above contextual factors into consideration, 
the author chose to focus on the faculty server as a sampling frame for course Web sites. 
The rationale being that this was the only server that housed and linked to the largest 
number of course Web sites offered by SU and SUNY-ESF. 

3.4 Methodological Approach 

A theoretical framework frequently cited in the literature on survey design is the 
Total Survey Design (Fowler, 1993; Platek, 1985). This framework guided the design of 
the study and operationalization of the research questions and data collection procedures. 
The following section summarizes contextual and procedural factors considered in light 
of this systemic theoretical framework. 

3.4.1 Choice of Methods 

It is the purpose of the study to survey a wide range of students from various 
disciplines to allow for larger variations across groups and subsequent generalizability of 
common student experiences in using various Web sites. Using a questionnaire to survey 
a large population was judged to be the most appropriate method. 
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3.4.2 Sampling Procedures 

The purpose of this section is to describe and justify how course Web sites and 
students were selected to participate in the study. 

3.4.2. 1 Web Site Sampling 

Sampling procedures began by selecting the Central New York (CNY) region for 
convenience of location and logistics for the author. At the time the survey was 
conducted, Syracuse University also had the largest pool of supplementary course Web 
sites located on one server in the region (based on author’s observations and several visits 
to WWW sites used in a similar context in the CNY region). Selection of course Web 
sites was conducted using the Syracuse University’s faculty server as the sampling frame. 
This server is accessible at http : //syllabus . syr. edu . After an initial screening of 174 Web 
sites for the fall of 1997, a total of 71 course Web sites were included in the sampling 
frame. Of the 71 course Web sites, 17 Web sites made the first screening by receiving a 
score of 3.5 [out of 9] or higher on the WIDEC instrument created by the author to screen 
the various course Web sites according to their instructional design quality. WIDEC 
consists of 30 items with a maximum score on a Likert scale of 90. This became the 
WIDEC score which was later converted to a GAGNE score by dividing 90 by 10 to 
make the scale comparable to the nine events. Of the 17 Web sites, 9 Web sites made the 
final screening: three Web sites with scores ranging from 3.5 - 4.4, 3 sites with scores 
ranging from 4. 5-5. 4, and three sites with scores ranging from 5. 5-8.0. 

3.4.2.2 Student Sampling 

Participants for the study were students taking classes that have Web sites 
posted on a dedicated server on SU campus during the fall of 1998. The sampling 
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frame for students was comprised of students in the classes that were included in 
the study. In classes with 25 students or less, each student received a 
questionnaire. Three of the nine classes belonged to this category (SPP 500, 

WRT 100, WRT 300). The remaining six classes all had at least 100 students 
registered. Only 40 questionnaires were distributed to each of these larger 
classes. Sampling in this case took place by counting every nth student 
(calculated by size of class divided by 40). A total of 920 students were enrolled 
in the nine classes, 30% of which were included in student sampling pool (285). 

The number of respondents was 142/285. See section 5.2 in chapter five for a 
break down by course. 

3.4.2. 3 Sample Representativeness 

Findings from this study may be generalizable to Syracuse University students 
attending school during the academic year 1998-1999. Table 3.1 is a comparison of 
demographics with figures reported on the online SU Student Enrollment Report for fall 
1998 (Institutional Facts and Figures, 2000). Figures in Table 3 are reported in 
percentages. Table 3 does not include SUNY-ESF student enrollment for the fall of 1998. 
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Table 3 



Comparative Matrix on Common Variables Between Sample Demographics and 
Syracuse University Fall 1998 Enrollment Report in Percentages 



Common Variables 


Study* 


Fall 1998 




Sample 


Enrollment 


Totals (Full-time and Part-time) 






Undergraduates 


94 


72 


Graduates 


6 


28 


Academic Class 






Freshmen 


25 


23 


Sophomores 


15 


23 


Juniors 


29 


23 


Seniors 


24 


24 


Other 


1 


6 


Graduate** 


8 


- 


Gender 






Female 


54 


54 


Male 


46 


46 


School or College (Undergraduates only) 


School of Architecture 


0.8 


3.5 


College of Arts & Sciences 


43.5 


29.9 


School of Education 


4 . 6 


4 . 4 


College of Engineering & Computer Science 


4 . 6 


7.9 


College of Human Development 


0.8 


6 


School of Information Studies (1ST) 


0.8 


4.3 


School of Management 


7.6 


11 


College of Nursing 


NA 


2.8 


Maxwell School of Citizenship and Public Affairs 


0.8 


NA 


School Public Communications 


13 


11.6 


School Social Work 


NA 


1.6 


VPA 


2 


16.8 


SUNY-ESF 


14 . 8 


NA 


Nationality 


Undergraduate American 


91.7 


96.4 


Non-American 


8.3 


3.6 


Graduate American 


100 


79.8 


Non-American 


0 


20.2 



* 4 missing responses not included 

** Graduates in this study are not representative of SU Graduate population 
Note 1 : Figures include rounding errors 

Note 2: Student Enrollment in International Study Abroad Programs and University College and 
branches is not included since none of these students were represented in the sample. 

Note 3: The study did not examine diversity in terms of ethnicity 
Note 4: NA indicates unavailability or inapplicability 
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A comparison between sample demographics and the SU Enrollment Survey 
Report for fall 1998 showed a high level of representativeness, particularly with regard to 
representation of gender, each academic class, nationality (international students were 
over-represented in the sample), and several schools including Arts and Sciences, 
Education, Management, Maxwell, and Public Communications. The study's small 
sample size (N=142) and the context in which these Web sites was developed and used 
does not allow for generalizability of findings to all the student population of Syracuse 
University at all times nor to all course Web sites offered at SU. Findings are more 
appropriately generalizable to a population of approximately 920 students enrolled in the 
specified nine classes during the fall of 1998. Findings about the Web site may only be 
generalizable to course Web sites made available through the faculty Web server during 
the fall of 1998. However, the findings are transferable to similar contexts where course 
Web sites are being used. The following observations are worth noting in relation to 
generalizability of findings to the two units of analysis in this study (a) students and (b) 
course Web sites: 
a) Student Representativeness: 

The study findings are not generalizable to the graduate student population due to 
small sample size of graduates represented in the sample. The findings about the course 
Web sites are not generalizable to undergraduate student population attending Syracuse 
University's College of Nursing and School of Social Work since none were represented 
in the sample. 

□ Similarly, the study under-represented the student population in the Schools of 
Architecture and Information Studies and Colleges of Engineering and Computer 
Science and Human Development. It is worth noting that these schools and 







colleges were not well represented, if at all, in the Web site sampling pool to 
begin with. 

b) Web site Representativeness 

□ The findings related to the SUNY-ESF course Web site are not generalizable to 
SUNY-ESF course Web sites since this is an atypical Web site in that it was the 
only SUNY-ESF functional Web site available on the faculty server at the time 
the data was collected. 

□ Students' perceptions of their course Web sites are limited to a context where 
these Web sites are offered as supplementary instructional resources rather than 
stand-alone or as part of online distance education. 

□ The course Web sites may no longer be representative of Web sites on the SU 
faculty server as this data was collected in the fall of 1998. Each course Web site 
continues to evolve every semester. 

3.4.2.4 Increasing Response Rate 

A low response rate in a survey frequently accounts for poor interpretation of 
final results, whereas an acceptable response rate for a general survey could range from 
30% (Fowler, 1993) to 50% (Dillman, 1978). In order to increase the response rate, great 
effort went into the design of the questionnaire and its administration following the 
guidelines of Dillman (1978). The following steps were undertaken to increase response 
rate: 

a) Questionnaire Design 

In terms of look and feel, the questionnaire was field-tested with three students 
representative of the sample. In addition, various bright colors were used to increase 
appeal and also to facilitate coding of data by class and keeping track of responses. A 
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picture of a papyrus bookmark, offered as a promised raffle prize, was placed on the 
cover page of the questionnaire. 
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b) Pilot Studies 

Several methods were field-tested to confirm the final method of data collection 
and administration including: e-mail, Web-based online forms, campus-mail, regular 
mail, and in class administration. It became evident that in-class administration yielded 
the highest response rates but had to be rejected because this necessitated taking 20-30 
minutes of class time. Online surveys using Web forms were found to be risky, 
unfamiliar to students, and an unreliable means of gathering data particularly when 
respondents are novice users of the technology or simply do not use it. Therefore, to 
avoid response bias of this nature, Web surveys were not used. During the pilot, e-mail 
attachments yielded a response rate of 23% (N=30). Many students were having 
problems opening the files and/or sending them back. However, e-mail was resorted to 
for one-to-one follow up communications. 

c) Administration 

The survey was handed to each student personally by the author in class to reinforce 
its importance. Students were requested to mail back the questionnaires, which were 
enclosed in campus-mail envelopes with a return address so as to make it more 
convenient for students and at no monetary cost. Students were personally greeted by 
the author as they walked into class coupled with a 10-minute introduction to the survey 
that highlighted their roles as pioneers in this experience (see Appendix K for transcript). 
Verbal endorsement of the research study by the course instructors added to their openly 
expressed eagerness to better serve their students' academic needs using the technology, 
also served to give importance to the study. 
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c) Follow-Up 



In spite of all the above, the initial response rate was only 37%^. As part of the 
follow-up efforts, several strategies were used. One-to-one e-mail follow-ups further 
established a connection between personal students' input and value of the research study. 
This was easy to do since students were initially requested to give their name and email 
address when they received a questionnaire. To avoid any problems with e-mail 
attachments experienced during the pilot study, students were informed that the author 
would be outside their lecture halls during a specified week so they could pick up another 
form, in case they had difficulty opening the attachment. The author was physically 
available during follow-up weeks near lecture halls collecting undelivered questionnaires, 
replacing lost ones, encouraging response from students who pointed out that they do not 
use the Web sites. Given the above experience in data collection, the impact of these 
efforts resulted in an initial 37% response rate that increased to 49.8% with a follow-up. 
Response rates are provided in Table 6 in chapter five. 

3.5 Instruments 

This section discusses the process of developing essential instruments for 
gathering data to answer the research questions. For each of these instruments, a brief 
summary is presented on the purpose, design and development, reliability and validity 
measures undertaken, as well as implementation procedures. The following is a listing of 
instruments developed by the author for this study. 

3.5.1 Web site Instructional Design Evaluation Checklist (WIDEC) 



* For comparative purposes, it is worth noting that a similar data collection method employed in 1 994 by SU- Center 
for Support of Teaching and Learning (CSTL) also generated a 37% response rate (personal communications with Dr. 
Barbara Yonai- Assistant Director of Evaluation and Research, March 1 5^^, 2000). 
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This is a 30-item checklist that was designed to screen course Web sites for final 
inclusion in the study. Three sets of courses were selected to reflect ranges of low, 
middle, and high in terms of instructional quality as determined by the nine events of 
instruction. The instrument was piloted and content validated. Interclass correlation 
during the pilot was (r=0.87). Details on the purpose, design, development, validation, 
and implementation of WIDEC can be found in [Appendix B]. 

3.5.2 Student Web site Perception Survey (SWPS) 

SWPS is a 60-item questionnaire that was designed to measure students' 
perceptions of various Web site functions. The questionnaire [see Appendix C] consists 
of 6 sections: Section I elicits time spent on various functions. Section II elicits students' 
satisfaction with the course Web sites using open-ended questions that elicit perceived 
barriers to and facilitators of use and overall impact of the WWW. Section III elicits 
students' perceptions of the quality of Web site functions. Section IV elicits students' 
perceptions of the usefulness of various functions. Section V measures students' self- 
rated Internet skills, and Section VI elicits demographic data and characteristics. SWPS 
was content and face-validated by (a) experts in survey design, (b) three doctoral students 
with experience in instrument design, and (c) four student representatives and non- 
representative of the student sample. Ambiguities, wording problems, redundancy, 
scaling, and layout were revised. Construct validity was also conducted using factorial 
analysis. Cronbach's Alpha test for reliability and consistency was performed. Details on 
the purpose, design, development, validation, and implementation of SWPS are described 
in chapters four and five. 
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3.5.3 Faculty Survey 

A 10- item open-ended questionnaire of purpose and expectations for having a 
course Web site was designed [Appendix D]. This instrument was piloted. Questions 
were revised to obtain the necessary information for triangulation purposes. Data from 
this survey was used to support the data or resolve any discrepancies in findings. 

3.6 Analysis Framework 

3.6.1 Summary of Factor Analysis 

A confirmatory factor analysis technique was considered in this study over an 
exploratory factor analysis teehnique given that the dimensions, constructs, and items 
were all developed by the author using theoretical principles. While confirmatory factor 
analysis is used to assess the fitness of the theoretical model to the data (Pedhazur & 
Schmelkin, 1991), conventional factor analysis in this study resulted in confirming 
several items in each grouping originally created and therefore more sophisticated 
confirmatory factor analysis procedures were not deemed necessary as suggested by 
DeVillis (1991). A varimax rotation was used as a simple orthogonal rotation method 
that yielded meaningful item groupings and strong, unambiguous loadings (DeVillis, 
1991). The following factors were found representative of the corresponding theoretical 
principles: 

V INTERACT = time on task for communication and interactions with material 
[Gagne’s events land 6, Chickering & Ehrmann's principles 1,2,3, and 5] (T) 

V PREPARE = the usefulness of functions that facilitate recall of prior learning, 
present new information, provide learner guidance, and enhance retention and transfer 
through enrichment and remediation [Gagne’s events 3, 4, 5, and 9] (U) 
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GUIDE = the usefulness of practice, elicitation, and feedback functions [Gagne’s 
events 5, 6, and 7, Chickering & Ehrmann's principle 4] (U) 

ATTITUDE = the overall usefulness of the course Web sites and the extent to which 
students agreed that instructors’ encouraged the class to use the Web sites (U) 
APPEAL = quality of visual and auditory elements and features that increase 
motivation for learning and are attention holding [Gagne’s event 1 and 3] (Q) 
PURPOSE = quality of functions pertaining to clearly stating objectives and 
purposes, organization and importance of material on Web sites [Gagne’s event 3 and 

4] (Q) 

SKILLS = skills commonly used to cormect to people or to other computers and 
accounts via e-mail, FTP, and Telnet [Flashlight Project Item Bank] (I) 

(T) denotes Time Spent 
(Q) denoted Quality 

(I) denotes Internet Skills 

(U) denotes Usefulness 

The above factors were constructed conceptually and empirically: conceptually, 
drawing upon theoretical frameworks provided primarily by Gagne, Briggs, & Wager 
(1992) in the nine events of instruction, and the seven principles of good teaching and 
learning practices using technology as lever (Chickering & Ehrmarm, 1996) and 
empirically using factor analysis and the mean of item factor loadings combined for each 
dimension represented. Table 4 shows a summary of dimensions with corresponding 
indicator measures, items for each indicator that loaded >0.60 in factor analysis, and 
Cronbach Alpha. 

Table 4 

A Summary Table of Dimensions and Corresponding Indicator Measures, Items, 
and Cronbach Alpha Values 
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Dimensions 


Corresponding 

Factor 

(Indicator) 


Corresponding Items in SWPS 
with Factor Loading >0.60 


Cronbach 

Alpha* 


Time Spent 


INTERACT 


Q1,Q2,Q3,Q5,Q8,Q9,Q10 


0.89 


Instructional 


APPEAL 


Q15,Q16,Q18,Q19, Q27 


0.80 


Quality 


PURPOSE 


Q20,Q22,Q23,Q25 


0.77 


Usefulness 


ATTITUDE 


Q33,Q34 


0.67 




PREPARE 


Q35,Q44,Q45,Q46 


0.78 




GUIDE 


Q37,Q38,Q39,Q40,Q41,Q42,Q43 


0.86 


Skills 


SKILLS 


Q49a,Q49c,Q49f 


0.70 



* Cronbich Alpha of 0.70 standard value or higher is indicative of established reliability (Nunnally, 1978). 



3.6.2 Index Construction 

The purpose of index construction was to create an overall score for each 
indicator (factor). Following recommendations from factor analysts (Pedhazur & 
Schmelkin, 1991), for each indicator, an arithmetic mean is computed for all item scores 
(rated by the respondent) belonging to that factor. It is worth noting that items with 
missing values were not included in the mean computation of the index score. Second, 
taking into consideration the differences in item factor loadings, each item was weighted 
by its factor loading in the arithmetic calculation of the mean. Substituting a mean score 
for the missing item value was rejected because it does not change the result. In other 
words: 

sum( item score*item factor loading) / number of items = respondent mean index score 
A hypothetical example is provided for PREPARE in Table 5: 

Table 5 
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Hypothetical Data Matrix® for PREPARE 





Q35 


Q44 


Q45 


Q46 


MEAN 


Respondent 1 


2(.63) 


2(.80) 


3(.83) 


2(.78) 


1.73 


Respondent 2 


1(.63) 


3(.80) 


2(.83) 


2(.78) 


1.41 


Respondent 3 


2(.63) 




4(.83) 


2(.78) 


2.1 



Hypothetical Mean Index Score for PREPARE 1 .75 

3.6.3 Data Coding and Cleaning and Missing Responses 



Data coding, entry, and analysis were conducted by the author using the SAS™ 
statistical package. Data coding primarily used the actual Likert-scales used in the 
instrument with the exception of the NA items that were coded as "9" and later treated as 
missing responses. Also in the Time Spent index, the half-hour was coded as "8" and 
later converted to represent (.5) of an hour to allow for calculations of percentages. Zeros 
on the Likert scale were coded as (0). Data cleaning followed recommendations from 
Babbie on possible-code cleaning and treatment of missing responses (1992). Possible- 
code cleaning was conducted by checking errors of data entry. Handling missing data is 
necessary in index construction (Babbie, 1990). Babbie recommended several 
procedures including: (a) careful analysis of data for possible interpretation, (b) 
excluding respondents with missing data, and (c) treating the missing items in the same 
way as the other items. Each of these procedures was considered and only one 
respondent was excluded altogether. No meaningful patterns were found amongst 
missing responses. In addition Not Applicable/Not Available/No Basis for Judgment 

® A similar table format in Tangmanee's (1999) dissertation study was adapted here for illustration 
purposes 
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(NA) and Don't Know (DK) responses were treated as missing values and were not 
included in computations except where noted. 

Qualitative and quantitative analysis methods. were used to analyze data. Analysis 
of narrative data from open-ended questions on SWPS was based on Miles and 
Huberman (1994) framework for qualitative data analysis and the quantitative numerical 
coding scheme described in Babbie (1989). The open-ended responses from the 
questionnaire were coded using categorization and analysis of data to identify emerging 
patterns and themes. Each response generated from the open-ended questions was coded 
by individual respondent and by course. Then patterns were identified and assigned a 
code. This code was then examined for its frequency of occurrence in relation to other 
codes using percentages. Quantitative data was analyzed using both descriptive and 
inferential statistics. Frequency distributions, percentages, means, standard deviations, 
were computed to obtain descriptive statistics. Further interpretation of the data required 
the use of statistical tests such as Pearson's product moment correlations, simple 
regression analysis, ANOVA, factor analysis and Cronbach's Alpha. 

3.7 Chapter Summary 

This chapter has delineated and discussed how the proposed study sought to 
document, analyze, and develop an understanding of college students’ reported 
experience of the use of the World Wide Web as an instructional resource. In this 
chapter, specific procedures and rationale for survey methods were discussed. Figure B 
depicts a flow-chart that summarizes data collection procedures. The proposed study 
employs a self-administered survey for data collection. An effort was made to develop a 
questionnaire that would yield reliable and valid responses. Measures were also taken to 




101 



85 



increase the response rate. Data were collected from nine classes offered during the fall 
of 1998 on the Syracuse University campus. These classes are cross-disciplined and 
therefore varied in terms of subject matter, level of difficulty, instructional methods 
employed, and student population characteristics. The intent was to increase variation in 
findings by sampling from a diverse pool of students in terms of their discipline of study 
and academic standing in addition to other demographics and other contextually relevant 
characteristics. Thereby, this further illuminates on current WWW technology integration 
efforts through empirical findings. 



Figure B A Flow Chart Illustrating Data Collection Procedures 



Start 

► List All Fall 1998 Course Web sites (SU) on syllbus.syr.edu 

^ Stratify Web sites using Phase I of Checklist 
p. Eliminate all sites that scored less than 3 

^ Evaluate Web sites using WEDEC (version validated by expert panel) 

p. Eliminate all Web sites scoring less than 3.5 

Stratified Sampling of 17 course scoring 3.5 and above = N= 9 

Administer SWPS to all students in class (section) attending the 9 
^ courses selected. 

p. Administer faculty survey to instructors of these selected sites . 
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CHAPTER FOUR 

The Development, Validation, and Implementation of 

SWPS 

4.1 Introduction 

As pointed out in chapter two, at the onset of this study there were only a handful 
of studies that examined students’ perceptions of instructional Web sites. As yet, there 
are no initiatives to study the phenomenon of campus-wide course Web sites used as 
instructional resources for campus-based courses. While this study is primarily a case 
study of one private university faculty initiatives and student perceptions, concern with a 
representation of students across and within disciplines led to the utilization of survey 
design methods. The Student Web site Perception Survey (SWPS) is a theory-based 
instrument designed to gauge students’ perceptions drawing upon a theoretical 
framework (see Figure A) based on principles of instructional design and 
recommendations from research on good teaching and learning practices using 
technology as a lever. In particular, SWPS attempts to gauge the instructional quality and 
usefulness of a range of course Web sites that are representative of multi-disciplines and 
are wide-ranging in sophistication of design. 

4.2 Purpose of SWPS 

SWPS was designed to facilitate students' reporting of their perceptions of their 
course Web sites [See Appendix C for questionnaire]. Data from one-hour interviews 
with students revealed limited expressions on behalf of students in describing their 
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experience in using the course Web sites. The instrument provides respondents with an 
elaborate set of guidelines for judgment of the value of the phenomenon of study. 

Elicited responses were guided by four dimensions: (a) time spent on certain Web site 
functions as indicator of use, (b) judgments of the instructional quality of functions, (c) 
perceptions of usefulness, and (d) barriers to and facilitators of use and overall impact on 
students' learning experiences. 

4.3 Design and Development of SWPS 

Initially, exploration of students' experiences with the WWW as a course resource 
began with the administration of a written questionnaire. This questionnaire was 
developed by the author after examination of several existing instruments developed 
including the Motivational Analysis Checklist (WebMAC ) (Small, 1997), Keller's 
Motivational Tactics Checklist (Keller, 1989), Flashlight Student Inventory Item 
Bank, (Flashlight Project, 1997), and WIDEC [See Appendix B]. It is worth noting that 
several pilot questionnaires were administered between 1996-1998 prior to the final pilot 
questionnaire administered in the fall 1998. These earlier versions assisted in refining 
and grounding the final research questions for the current Student Web site Perceptions 
Survey (SWPS). The research questions provided the framework for designing the 
questionnaire. SWPS consisted of several forced-choice questions and four open-ended 
questions. As stated above, several dimensions guided the operationalization of these 
research questions that were represented in the questionnaire in six sections: The forced- 
choice items were measured using likert-type scale ranging from 0-7. Each section is 
summarized below in Table 6 and described next in greater detail. 
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Table 6 

Summary of the Dimension, Outcome Measures, and Corresponding Items in SWPS 



Dimension by 
Section in SWPS 


Outcome Measures 


Corresponding 
Items in 
SWPS* 


I - Time Spent (T) 


Time Spent on various Web course functions 


Ql-Qll 


II - Satisfaction (S) 


3 Greatest barriers to use of Web site 


Q12 


3 Greatest help to use of Web site 


Q13 


Perceived impact of WWW on learning experience 


Q14 


Overall comments on use of Web site 


Q60 


III- Instructional 
Quality (Q) 


Perceived appeal of motivational functions 


Q15-Q20 


Perceived PURPOSE of statement of Web site 
objectives 


Q21-Q23 Q30 


Perceived organization of presentation of 
information 


Q24-Q29 


Perceived support provided by enriching and 
remediating functions 


Q3I-Q32 


III Usefulness (U) 


Perceived usefulness of various functions on the 
Web site including recall, performance elicitation, 
practice, and feedback, enhancing retention and 
transfer 


Q33-Q47 


IV- Internet Skills (I) 


e-mail, WWW searches, FTP, Telnet, Programming, 
years of access, and creating Web pages 


Q49-Q48 


VI - Demographics 
(D) 


Class, age, sex, age, GPA, nationality, major, etc.. 


Q50-Q54, Q59 


VI - Characteristics 
(C) 


Access to computers, modem speed, type of 
connectivity, residential proximity, reasons for 
taking course, distance, etc. 


Q55-Q58 
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Section I - Time Spent (T) - Students reported time spent on specific functions on the 
Web sites. 

Section I in the questionnaire attempted to measure the time allocated on a 
typical week to specific functions provided on the Web site. A 7-point Likert-type scale 
was used to measure perceived time spent in hours per week over a one-semester period 
of time. This section addresses time spent on task and overall use of specific functions 
on the Web sites. This question was considered important given that it has been 
previously reported that use of technology in higher education is on the rise, for college 
instruction in particular (Green, 1996). Items for this index were drawn and adapted 
from the Flashlight Student Inventory Item Bank (Flashlight Project, 1997). Content 
validation for this section of the SWPS was initially based on that reported for the 
Flashlight Projects' Current Student Inventory (CSI). 

According to the project handbook (1998) the CSI was subjected to content 
validity testing. Initially, 2000 students responded to the first version of CSI. This was 
followed by an 18-month series of focus groups with students, faculty, and 
administrators from the five Flashlight participating higher education institutions. 
Meetings were held to discuss the extent to which respondents liked the items and 
agreed upon their interpretation. Items were revised to correct any ambiguity problems, 
and to include suggestions made toward their improvement. Based on Flashlight 
recommendations, when creating the survey for this study, care was taken to retain as 
much of the wording of the items from the CSI as possible. However the order and 
context in which they were administered, were changed by the author and therefore 
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validity was compromised in this respect. However, a factorial analysis was computed 
on these items to establish construct validity. 

Section II - Satisfaction (S) - Students' perceived satisfaction with course web site 

The second section of the questiormaire consisted of four open-ended questions 
which attempted to tap into the feelings and thoughts of students as they engage in this 
learning experience. Student satisfaction is a necessary criterion for effective instruction 
(Dick & Reiser, 1989; Keller, 1987a, 1987b, 1987c). The open-ended questions were 
designed to elicit students’ perceived satisfaction with the technology in terms of overall 
opinion, the three greatest barriers to the use of the course Web site, the three most 
helpful factors that contributed to use of the course Web site, and the impact of the 
technology on their learning experiences. The section addresses the question 
investigating influential factors on students’ perceptions. 

Section III - Instructional Quality (Q)- Students' perception of instructional quality of 
specific functions on the Web sites. 

Section three of the questiormaire examines the complexities instructional quality 
of the Web site using evaluation criteria drawn from these principles of instructional 
design. This section addresses the perceived quality question in the study that explores 
students' perceptions of the quality of certain functionalities of their course Web sites. 
Items for this index were created using WIDEC items based on Gagne, Briggs, and 
Wager's (1992) nine events of instruction as use and reworded to suit student language 
[Appendix B - 2 lists Gagne’s nine events and corresponding WIDEC and SWPS Items] 
Section IV - Usefulness (U) - Students' perceived usefulness of certain instructional 
activities on the course Web sites. 
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4.4 Validating SWPS 

Validation of the instrument will primarily address content and construct validity 
(Huck and Cormier, 1996). Drafting the survey following several recommendations 
(Babbie, 1992; Dillman, 1978) in order to attract a survey respondent and thereby 
enhance reliability and validity of questionnaire items. These recommendations also 
increase the response rate. Content and face validation was achieved through various 
strategies: field-testing, pilot-testing, and expert reviews. Construct validation and 
internal consistency of items in each category were achieved using statistical tests such as 
Factor Analysis.and Cronbach's Alpha respectively. Next, is a summary of these 
procedures for validation. 

4.4.1 Statistical Testing 

Huck and Cormier (1996) also recommend factor analysis as suitable for 
construct validation. Factor analysis is a powerful tool of construct validation (Kerlinger, 
1986). Factor analysis was used for construct validation purposes and to derive outcome 
measures to further operationalize four dimensions from the above listed in Table: Time 
Spent (T), Instructional Quality (Q), Usefulness (U), and Internet skills (I). 

For each of these four constructs, only items that loaded (>0.60) were selected 
and labeled to reflect the combined characteristics of the items loadings. Factor analysts 
suggest that only a factor whose eigenvalue is greater than one (>1) be extracted and 
retained for the next steps of factor analysis (Harman, 1976). While factor loadings of 
0.40 to 0.50 are considered acceptable (Pedhazur & Schmelkin, 1991), in this study, this 
standard seemed low by comparison of other item factor loadings. Theoretical 
underpinnings were drawn upon to further explain item loading on any of these factors. 
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Particularly, the nine events (Gagne, Briggs, & Wager, 1992) and the seven principles of 
good teaching practices using technology as lever (Chickering & Ehrmarm, 1996) were 
used to match items for each derived measure with a corresponding event where possible. 
Table 7 shows item factor loadings, eigenvalues, communality estimates, and variance 
percentages, and Cronbach Alpha coefficients for each category of items designed to 
represent each dimension. Each of the derived factors were subsequently used for 
analysis of data and are summarized in section 3.6 and were discussed in chapter five. 
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Table 7 

Summary of Items and Factor Loadings for Varimax Orthagonal Two-Factor Solution for Ql-QlO 
(N=142) With Coefficient Alpha, Eigenvalues, Percentages of Variance for Factors 

Time Spent 

Coefficient Alpha* = 0.85 (standardized) 

Item Factor Loadings Communality 



1 2 ** 
INTERACT 



Q1 


0. 


66 


0 


. 05 


0. 


, 43 


Q2 


0. 


83 


0 


. 13 


0. 


, 71 


Q3 


0. 


67 


0 


. 16 


0. 


, 47 


Q4 


0. 


37 


0 


. 42 


0. 


, 32 


Q5 


0. 


79 


0 


.21 


0. 


, 67 


Q6 


-0 


'.05 


0. 


76 


0, 


, 58 


Q7 


0. 


00 


-0 


. 63 


0. 


. 39 


Q8 


0. 


84 


-0 


. 11 


0. 


, 71 


Q9 


0. 


86 


0 


o 

o 


0. 


, 74 


QIO 


0. 


85 


0 


. 15 


0. 


, 74 



Eigenvalue 4.5 1.2 

% of 4.5 1.3 

Variance 

*Coefficient Alpha for all items ** This factor was not used due to lack of conceptual meaningfiilness 
Notel: Boldface indicates highest factor loadings. 

Note 2: Item description is available in Appendix SWPS 

(Table 7 continues) 
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Table 7 Continued 

Summary of Items and Factor Loadings for Varimax Orthogonal Two-Factor Solution for Q15-Q32 
(N=142) With Coefficient Alpha, Eigenvalues, Percentages of Variance for Factors 

Instructional Quality 

Coefficient Alpha* = 0.90 (standardized) 

Item Factor Loadings Commonality 



1 2 
APPEAL PURPOSE 



Q15 


0, 


.85 


-0. 


. 24 


0, 


. 77 


Q16 


0, 


.69 


-0. 


. 04 


0, 


. 85 


Q17 


0. 


, 10 


0. 


. 05 


0, 


. 73 


Q18 


0, 


.82 


0. 


. 12 


0, 


.79 


Q19 


0, 


.75 


0. 


,22 


0, 


. 62 


Q20 


0. 


. 08 


0. 


.76 


0, 


. 59 


Q21 


0. 


, 27 


0. 


, 54 


0. 


, 67 


Q22 


0. 


, 05 


0. 


.73 


0. 


, 62 


Q23 


0. 


, 04 


0. 


.60 


0. 


, 72 


Q24 


0. 


, 57 


0. 


, 50 


0. 


, 72 


Q25 


0. 


, 05 


0. 


,76 


0. 


, 69 


Q2 6 


0. 


, 39 


0. 


, 54 


0. 


, 68 


Q27 


0. 


.79 


0. 


, 27 


0. 


,75 


Q28 


-0. 


, 00 


0. 


, 31 


0. 


, 72 


Q29 


0. 


, 07 


0. 


, 55 


0. 


, 65 


Q30 


0. 


, 17 


0. 


,35 


0. 


, 72 


Q31 


0. 


, 08 


0. 


, 00 


0. 


,78 


Q32 


-0. 


, 02 


0. 


, 09 


0. 


,78 



Eigenvalue 5.9 


2.8 


% of 3.7 


3.6 



Variance 



*Coefficient Alpha for all items 

Notel: Boldface indicates highest factor loadings. 

Note 2: Item description is available in Appendix SWPS 

(Table 7 continues) 
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Table 7 (Continued) 

Summary of Items and Factor Loadings for Varimax Orthogonal Three-Factor 
Solution for Q33-Q47 (N=142) With Coefficient Alpha, Eigenvalues, Percentages of 
Variance for Factors 

Usefulness 

Coefficient Alpha* = 0.89 (standardized) 



Item 




Factor Loadings 




Communality 




1 


2 


3 






GUIDE 


PREPARE 


ATTITUDE 




Q33 


-0.22 


0 .16 


0.89 


0 . 94 


Q34 


0 . 14 


0.12 


0.80 


0.90 


Q35 


0.51 


0.63 


-0.03 


0.75 


Q36 


-0 .06 


0.24 


0.0 


0 . 98 


Q37 


0.85 


0.14 


-0.17 


0.82 


Q38 


0.76 


0.52 


0.08 


0 .86 


Q3 9 


0.93 


0.27 


-0.07 


0 . 97 


Q40 


0.90 


0.08 


-0.07 


0 . 93 


Q41 


0.91 


-0 . 06 


0.06 


0 .89 


Q42 


0.81 


0.09 


0.40 


0 . 95 


Q43 


0.77 


0.33 


0.46 


0 . 92 


Q4 4 


0.25 


0.80 


0.41 


0 . 98 


Q45 


0.27 


0.83 


0.29 


0 . 90 


Q46 


0.29 


0.78 


0.35 


0 . 83 


Q47 


-0.21 


0 . 14 


0 . 39 


0 . 78 


Eigenvalue 


7 . 3507 


2 . 8965 


1.7647 




% of 


5.7 


3.8 


2 . 3 





Variance 



*Coefficient Alpha for all items 

Notel: Boldface indicates highest factor loading of item. 
Note 2: Item description is available in Appendix SWPS 




(Table 7 continues) 
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Table 7 (Continued) 

Summary of Items and Factor Loadings for Varimax Orthogonal Three-Factor 
Solution for Q48-Q49a-f (N=142) With Coefficient Alpha, Eigenvalues, Percentages 
of Variance for Factors 

Internet Skills 

Coefficient Alpha* = 0.89 (standardized) 



Item 


Factor Loadings 


Communality 




1 


2++ 






SKILLS 






Q48 


-0.03 


0.62 


0.38 


Q49a 


0.72 


0.17 


0.54 


Q49b** 


0.48 


0.52 


0.50 


Q49c 


0.76 


0.25 


0.64 


Q49d 


0.29 


0.69 


0.56 


Q49e** 


0.52 


0.47 


0.50 


Q49f 


0.68 


-0.46 


0.68 


Eigenvalue 


7 . 3507 


2.8965 




% of 


5.7 


3.8 





Variance 



*Coefficient Alpha for all items 

** These items were excluded due to low factor loadings 
++ This factor was ignored because it has no conceptual meaning' 
Notel: Boldface indicates highest factor loadings of item 
Note 2: Item description is available in Appendix SWPS 



Based on the above results of the factor analysis and the Alpha Coefficient of 
internal consistency, the following conclusions were drawn: 

□ Items with low factor loadings (<0.60) were ignored and removed from subsequent 
analysis. 

□ Conceptually derived constructs (e.g. PRESENT and ENRICH above) that were not 
confirmed by factor analysis, were not used in subsequent analysis even though the 
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internal consistency of the items was consistently high (ranging from 0.75-0.90). 
According to Babbie (1992), one of the disadvantages of factor analysis is that factors 
can be generated on a group of disparate variables without regard to substantive 
meaning (1992). Given the theoretical grounding of this factor analysis provided the 
rationale for discarding. Similarly, the author would rather not undermine the 
strength of theoretically derived and empirically tested constructs and recognizes the 
usefulness of subsequent revisions of the instrument by increasing the number of 
items and ultimately their power. 

4.4.2 Reliability Testing 

As mentioned above, reliability is the degree of consistency of measured 
performance across repeated testings (Huck & Cormier, 1996). While, Huck and 
Cormier (1996) recommend several approaches to reliability such as the coefficient of 
stability (Test-Retest) or the Alternate-Forms reliability, both these types do not seem 
feasible for this study. The reliability coefficient or the coefficient of internal consistency 
of the questionnaire items was considered a more appropriate test. According to Huck 
and Cormier (1996), instead of focusing on stability across time or on equivalence across 
forms, the test of internal consistency assesses the degree to which the items in the 
instrument are internally consistent. Internal consistency is defined as: 

. . .consistency across the parts of a measuring instrument, with the “parts” 
being individual questions or subsets of questions. To the extent that these 
parts “hang together” and measure the same thing, the full instrument is 
said to possess high internal consistency reliability, (p. 78-79) 
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Similarly, Pedhazur and Schmelkin (1991) refer to internal consistency reliability 
as to how consistent the questionnaire items capture the variation in the variable. While 
there are several other procedures, this test is best suited for the study because it is more 
versatile than any of the others in that it can be used with instruments made up of items 
that can be scored with three or more possible values (Likert-type scales). The other 
procedures are more favorable when using dichotomous values. Cronbach's Alpha is a 
measure of internal consistency for items that are not scored dichotomously (Borg and 
Gall, 1983). Since the questionnaire employed a four-point to seven-point Likert-type 
scale, Cronbach's Alpha could therefore be used to determine the reliability of these 
scales. 

Table 8 illustrates the reliability rate for items belonging to the four dimensions: 
Time Spent, Instructional Quality, Usefulness, and Internet Skills. Each of the derived 
construct measure from the factor analysis presented above was also tested for reliability. 
For exploratory research such as this study, traditionally, a Coefficient Alpha's in range 
of (0.70) is considered acceptable. In this study, almost all but the ATTITUDE had 
Alpha coefficients higher than the 0.70. The ATTITUDE index (0.67) was not discarded 
nonetheless, given the close to 0.70 score and the non-life threatening nature of the 
phenomenon understudy (Babbie, 1992, Pedhazur & Schmelkin, 1991). Table 18 also 
depicts results for items that were conceptually grouped based on the constructs stated 
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Table 8 

Results Of Cronbach's Coefficient Alpha Reliability Test for Items Belonging to 4 
Main Dimensions: Time Spent, Quality, Usefulness, and Internet Skills and Items 
Belonging to their Corresponding Indicators 



Dimension 

Indicator 


Items 


Cronbach Alpha* 


Time Spent 


Qi-Qm A- 


0.85 fe 


INTERACT 


Q1,Q2,Q3,Q5,Q8,Q9,Q10 


0.89 


Quality . ^ 


Q15-Q32 


0.90 


APPEAL 


Q15,Q16,Q17,Q18,Q19,Q20 


0.73 


PURPOSE 


Q21,Q22,Q23,Q30 


0.80 


PRESENT** 


Q24,Q25,Q26,Q27,Q28,Q29 


0.80 


ENRICH** 


Q31,Q32 


0.75 


Usefulness 


Q33-Q47 


0.89 


ATTITUDE 


Q33,Q34 


0.67 


PREPARE 


Q35,Q44,Q45,Q46 


0.78 


GUIDE 


Q37,Q38 

Q39,Q40,Q41,Q42,Q43 


0.86 


Internet Skills 


Q48-Q49(a-f) 


0.76 


SKILLS 


Q49a,Q49c,Q49f 


0.70 



* Cronbach Alpha of 0.70 standard value or higher is indicative of established reliability. 

** Two derived variables based on theory, originally created in SWPS but were not confirmed by factor 
analysis so were excluded from subsequent analysis. 

d) Note: Shaded area shows alpha for all items included in SWPS by Category 
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4.4.3 Field Testing 

Field testing the Questionnaire constitutes, according to Dillman (1978), a pre-test 
that includes respondents from three distinct groups. These three are (a) those with 
similar training as the author conducting the survey; (b) those with substantive 
knowledge in survey or subject matter; and (c) those with similar skills or conditions as 
those of the population under study. To begin with, the author’s dissertation committee 
reviewed the survey based on their expert judgment as having both methodological and 
subject-matter expertise. In addition to several classmates and colleagues, four experts in 
survey research methods, statistics, and content were consulted on the face and content 
validity of the questioimaire [see Appendix C - 1 for letter of Reviewers of SWPS]. 

Four students representing the age among other population demographics of the 
intended student sample were selected from campus to respond to the instrument. Three 
of these students were interns at SU but went to schools other than SU and their reviews 
helped to clarify any contextual ambiguities. A half-hour debriefing session followed 
after their completion of SWPS. Structured interview questions that were asked of the 
respondent during the debriefing session. The purpose of the debriefing session is to 
identify any ambiguity, problems, and suggestions for improvement. The form used for 
debriefing sessions for both experts and students is in [Appendix C - 2]. 

4.4.4 Pilot Testing 

Finally, as a culminating activity in the validation process, a pilot test was conducted 
during the summer with two classes that were offered during the summer 1998. One is 
an online course in Psychology while the other is a course on the Internet for Educators. 
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Student enrollment is approximately 25 for each class. In both cases the questionnaire 
was mailed to the students. The Psychology group received it online via e-mail and 
returned it in the same manner. The Internet group received their survey and returned it 
by postal services. There were several shortcomings for this pilot that the author became 
aware of and took into consideration when administering SWPS. 

One of the shortcomings of the summer pilot study was that it was conducted in 
the summer and student demographics changes significantly from the typical one during 
the academic year. Several students in this sample were staff employees or non-degree 
students. Secondly, the courses themselves were typical summer courses that were 
designed to attract a typical student population. Other confounding variables were the 
method by which data was administered and collected. The Internet group received the 
questionnaire through postal delivery in a self-addressed envelope, while the Psychology 
group communicated through electronic mail. This difference did not affect the response 
rate as much as was expected. Both methods demanded a lot of work in terms of follow 
up. However, this experience confirmed for the author that campus-mail, which was used 
in an earlier pilot study, was the most efficient, reliable, and cost-effective method of data 
collection. 

Besides, the timing of data administration was also considered. The author 
learned that the survey should not be administered around exam periods or toward the 
end of the semester. For example, the Internet class received the questionnaire after they 
had completed the class, whereas the Psychology class received the survey on the last 
week before class was over. Follow up phone calls to the Internet class showed that the 
time lapse might have made a difference in terms of recall and reaction. Hence, it is 
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better that one sends out the survey before classes are over. Finally, another crucial 
variable to consider is the negotiation that takes place with the course instructor prior to 
administering survey. Some faculty preferred to determine the maimer and the time 
period when the survey should be administered. While this makes it more convenient 
and perhaps more rewarding in the long term, it does confound standardization of survey 
administration. In the pilot, both instructors endorsed the survey with a letter addressed 
either personally to the student, or through the class listserv. 

4.5 Administering SWPS 

A letter of invitation was first sent out to the course instructor inviting him or her 
to participate in the study and to allow his/her class to be included in the sample in 
[Appendix C - 3 ]. Final, administration of the student questionnaire was conducted by 
the author. Once the classes to be included in the sample were identified, the author e- 
mailed the course instructor and sought permission using the faculty consent form in 
[Appendix C - 4]. The questionnaire was distributed to all students attending class on that 
day. Each questionnaire and an informed consent form were enclosed in campus-mail 
envelope with a return address. Students were requested to mail them within two weeks. 

Students sampled were given a questionnaire as they approached the lecture hall. 
The author would be standing at the entrance, as students filtered in. In cases where there 
was more than one entrance to the lecture hall, the author requested assistance from a 
friend to co-distribute the questionnaire after having received instructions accordingly. 
Students generally tended to gradually filter in singles, pairs or groups of three, which 
made the administration process manageable. As students filtered in, care was taken to 
briefly inform students that this was a survey sample not related to any course 
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requirement or activity. This was important since some students did want to know if they 
should have received a questiormaire or were missing something related to class. The 
author also had to make judgment calls in terms of not distributing questiormaires to 
students who informed us that they already received SWPS in another class. In this 
situation, questiormaires were re-issued to close-by alternates. This also was the case 
when some students said that they were too busy at that time, or the one or two who 
happened to leave their questiormaires behind after class or simply dropped them. The 
author always stayed until the end of class, because the last 10 minutes were given up by 
the instructor for the introduction to the study and purpose of the questiormaire, and 
directions on where to send the questiormaire. On some occasions, however, the 10 
minutes were at the begirming of class. 

[Appendix C - 5] contains a copy of the transcription read out to students. 

4.5.1 Timeline for Data Collection 

Administration of SWPS was scheduled on the last week of October 1998 and first 
week of November 1998 after having taken into consideration the following: 

a. The last date to drop and add a course had passed; 

b. Mid-term Exams were over for each class scheduled; 

c. This time of the semester was timely for data collection based on negotiations 
between researcher and course instructors on where students are in their learning 
process. 
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4.5.2 Obtaining Informed Consent 

Consent was obtained from three main entities: the Syracuse University 

Institutional Review Board (IRB), the faculty who taught the classes participating in the 

study, and the participants themselves. The following steps were undertaken: 

1 . The Institutional Review Board at Syracuse University had been notified, necessary 
documents submitted, and approval was obtained [See Appendix C - 6]. 

2. Consent was also requested from faculty offering these classes. An e-mail request 
was sent out to each faculty member and a subsequent follow-up letter on official 
departmental letterhead was mailed through campus. An office visit was also made 
for those who request it and a copy of the questionnaire was sent to those who 
expressed an interest in seeing it. Follow-up e-mails, phone calls, and/or office visits 
were made to ensure consent was obtained and the procedure for data collection was 
not considered intrusive to the faculty [See Appendix C - 4 for Faculty Consent 
Form]. 

3. Participant consent forms were handed out with the questionnaire for students to sign 
[Appendix C - 7]. 

4. A “Thank You” card was sent to each participating faculty a week after data 
collection. 

5. A papyrus painting was presented to each participating faculty and TA as a token of 
appreciation for their support and encouragement. 

6. A "Thank You" note was sent out to all students through a class listserv or on their 
course Web site. A papyrus bookmark was sent out to each student who participated 




in the survey. 
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7. A summary report of findings from the study will be mailed to each participating 
faculty member upon their request. 

4.6 Chapter Summary 

This chapter described the procedures involved in designing and developing a 
questionnaire. SWPS is a theory-based instrument designed to measure primarily the 
usefulness and instructional quality of course Web sites. In addition, the instrument also 
gauges time spent by students on individual functions; barriers to and facilitators of use 
of course Web sites; and overall impact as perceived by students using these course Web 
sites. The purpose of the questionnaire was to facilitate students’ reporting of their 
perceptions of their learning experiences of their use (or lack) of their course Web sites. 
SWPS was construct validated using factor analysis which resulted in confirming five 
constructs where item factor loading >0.60. Item consistency was measured for 
reliability using Cronbach’s Alpha where item clusters had a range of r=0.085 - 0.90. 
Details of instrument field and pilot testing, in addition to administration procedures and 
timelines brought chapter five to a close. 
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CHAPTER FIVE 

Results 

5.1 Introduction 

This chapter presents the results that answer the questions: (1) How did students 
allocate their time using their course Web sites? (2) What was the quality of the course 
Web sites according to students? (3) How useful did students perceive their course Web 
sites? (4) What demographics and characteristics influenced students’ use and 
perceptions of quality and usefulness of course Web sites? (5) What were some student 
reported barriers to and facilitators of use of the course Web sites and overall impact of 
the WWW on students' learning experiences? 

The first section of this chapter presents a summary of response rates. The second 
section describes the demographics and characteristics of the study sample. The third 
section presents data regarding students' reported use of various functions of their course 
Web sites on the Time Spent (T) dimension represented by INTERACT. This includes 
interaction with materials on the course Web sites and communication functions (peers, 
faculty, teaching assistants, and other experts). Students' judgments of the quality of the 
course Web sites are reported in the fourth section. Quality is represented by two 
indicators. The first pertains to appeal and motivation for learning (APPEAL). The 
second pertains to clarity of objectives and Web sites' purposes, organization of course 
Web sites, and importance of materials on the Web sites (PURPOSE). These indicators 
represent the Instructional Quality dimension (Q). The fifth section reports students’ 
perceptions of the usefulness by three indicators: a) functions including online practice. 
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elicitation, and feedback (GUIDE); b) functions that facilitate recall of prior learning and 
enhance retention and transfer through remediation and emichment (PREPARE); and c) 
students’ overall attitude toward the usefulness of their course Web sites including their 
perception of instructors’ encouragement to use a course Web site (ATTITUDE). These 
three indexes represent the Usefulness dimension (U). The sixth section presents findings 
of statistical relationships among various demographic variables and characteristics 
including the (SKILLS) index. The results provide insights into student demographics 
and characteristics that may have influenced students' perceptions of their usefulness and 
quality of their course Web sites and the time spent on various functions. The final 
section of this chapter presents summary findings of the narrative open-ended data 
addressing students Satisfaction (S) with overall experience within the context of 
perceived impact, barriers to and facilitators of use of the course Web sites that serve to 
contextualize interpretations of the above findings. 

The data were analyzed statistically using the SAS™ package. The level of 
significance, in all cases throughout the analysis, was set at a minimum alpha level of 
0.05. Pearson product moment coefficient and various descriptive statistics such as 
frequency distribution, means, standard deviations and percentages were used to analyze 
and report the data. Inferential statistics were also computed including ANOVA and 
simple regression. 

5.2 Response Rate Results 

Data were collected during the fall of 1998. The study sample consisted of 142 
respondents drawn from nine on-campus courses that had a course Web site. 
Approximately 30% of the total student population registered for these 17 classes 
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(N=920) were sampled from the 9 classes that made the final WIDEC screening. Of this 
sample (N=285), 49.8% responded (N= 142). As explained earlier in section 3.4.2.2, in 
large classes, 40 questionnaires were handed out to students at the classroom entrance. 
Every nth student entering the class was handed a questionnaire. Variations of n occurred 
based on calculations of total number of students divided by 40. In smaller classes, all 
students attending class that day received a questionnaire. Table 9 categorizes the 
response rate in frequencies and percentages. 
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Table 9 

Response Rates in Percentages by Course and for Total Sample’ 



Course 

ID* * 


Course Title 


Student 
Sample Size 
(in relation to 
total class 
size)* 


/ 


% of 

Response 

Per 

Course 


%of 
Response 
for Total 
Sample 


BIO 400 


Immunobiology 


40/89 


28 


70 


19.7 


ECN 200 


Economic Ideas and Issues 


40/99 


13 


32.5 


9.2 


FOR 300 


Introduction to Soils 


40/103 


21 


52.5 


14.8 


HST 100 


Introduction to American History 


40/185 


15 


37.5 


10.6 


PAF 100 


Introduction to Public Affairs 


40/151 


21 


52.5 


14.8 


PHY 200 


General Physics 


40/240 


14 


35 


9.9 


SPP 500 


Phonation Disorders 


20/21 


7 


35 


4.9 


WRTIOO 


Freshmen Writing 


14/14 


13 


92.9 


9.2 


WRT300 


Studio Writing 


11/18 


10 


90.9 


7.0 


Total 




285/920 


142 




100% 



* Course ID numbers changed for anonymity 



5.3 Summary of Student Demographics and Characteristics 

According to Babbie (1992), knowledge of the respondents is a necessary step 



prior to conducting analysis because this provides the context for performing the main 



analysis and for interpretations of the results. [Appendix F includes summary tables and 



detailed narrative of data characteristics and demographics]. A typical student of the 



study sample would be a 20 year old American female, of junior academic standing with 



a GPA of 3.3. She is most probably taking the course as a requirement. She lives on 



Syracuse University’s North Campus. 



She has access to a computer on a regular basis and uses campus terminals for 



Internet access through an Ethernet connection rather than dial-in. She has been using 



the WWW for two to three years. 




’ All numbers in % are rounded off to the nearest decimal point. 

* Total class size figures are based on official school registration records and not actual number of students 
attending on day of data collection. 
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She considers her Internet skills overall to be low to medium. More specifically, 
her best skills are in using e-mail, followed closely by medium to high skills in searching 
the WWW. She has low to medium skills in the use of telnet and FTP. Her lowest skills 
are in programming and creating or editing Web pages. 

5.4 Question # 1 

How did students allocate their time using their course Web sites? 

This section presents and discusses findings regarding percentage of users and 
non-users of various functions of the course Web sites (specifically, the amount of time 
students allocate to each given function in a typical week). In order to answer this 
question, a Time Spent category (Questions 1-10, Alpha = 0.85, and 1 1 as open-ended) 
was constructed from these items included in the student Web site perception survey 
(SWPS, see Appendix C). Using a 7-point Likert-type scale (NA, 0, 1/2 hour, 1, 2, 3, 44 - 
hours), students were asked to report how much time they spent on using their course 
Web sites, in a typical week during the semester. Items for this section of SWPS were 
drawn from the Flashlight CSI section relevant to Technology Use, which are based on 
the seven principles as the theoretical basis. INTERACT was confirmed by factor 
analysis and used to create indexes for Time Spent (T) (see section 3.6.2 for index 
construction). The items are listed in Table 10. 

5.4.1 Student Reported Use of Web site Functions 

Table 10 depicts results in percentages of users versus non-users of these 
functions in rank order by percentages. As can be seen from Table 10 below, there were 
two main categories of functions used by a large proportion of students: (a) interacting 
with material and (b) communicating with peers, faculty, and teaching assistants. 
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Table 10 

Percentage of Users and Non-Users per Function in Rank Order by Users 



Function 


n 


% 

Users 


% 

Non- 

Users 


% 

Who Stated 
Function N/A 


Q1 . Searching the WWW to access reference 

materials and/or conduct research for the course. 


141^” 


78 


17 


5 


Q2. Downloading/printing information from the 
course Web site. 


142 


72.6 


24.6 


2.8 


Q5. Picking up or completing assignments on 
the course Web site. 


142 


58.1 


31 


9.9 


Q9. Communicating with the course instructor/ 
teaching assistant(s), via e-mail, to make 
a suggestion/ complain/ask questions about 
the course. 


142 


52.8 


43.7 


3.5 


Q3. Viewing a multi-media presentation/lecture 
notes on the course Web site. 


142 


45.8 


36.6 


17.6 


Q4. Viewing slides or pictures on the course Web site. 


142 


39.5 


37.3 


23.2 


Q8. Communicating with other students, 

via e-mail/listserv/hypermail/Chat, etc. to 
discuss the ideas and concepts taught in the course. 


142 


38.7 


51.4 


9.9 


QIO. Communicating with content experts outside 
your college or University via e-mail or 
through the course Web site via hypermail 
or Chat. 


142 


15.2 


67.6 


16.2 


Q6. Taking an online quiz through the course 
Web site. 


142 


14.8 


40.8 


44.4 


Q7. Creating or updating a home page on the WWW. 


142 


8.2 


51.4 


39.4 



5.4.1 .1 Functions with the highest percentages of users 

The function with the highest percentage of users is searching the WWW to 

access reference materials and/or conduct research for the course (Ql,78%). The second 
function with the largest percentage of users is downloading/printing information from 
the course Web site (Q2, 72.6%). Furthermore, two other functions were used by more 
than half the sample. Picking up or completing assignments on the course Web site 
ranked as the third function with highest usage (Q5, 58.1%) and communicating with the 
course instructor/Teaching assistant(s), via e-mail, to make a suggestion/complain/ask 
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questions about the course (Q9, 52.8%). These communication functions illustrate 
interactive use of the course Web site. These findings may support the claim that the 
WWW has interactive capabilities that particularly facilitate communication amongst 
groups (Dills & Romiszowski, 1997; Ehrmann, 1997a; Khan, 1997; Turoff, 1997). These 
findings also support expectations of the strengths and attributes of the WWW as a 
research tool given that it is currently the largest electronic repository of information. 

5.4.1 .2 Functions with the highest percentages of non-users 

The functions that appeared to have been least used were taking an online Quiz 

through the course Web site (Q6, 14.8%) and creating or updating a home page on the 
WWW (Q7, 8.2%). A closer examination of the data, however, showed that these two 
functions were perceived as unavailable by the largest two student proportions whereby 
44.4% did not have the opportunity of taking online quizzes and 39.4% did not have to 
create or update a Web Page. Results may have differed had they been made available on 
more course Web sites. Furthermore, these items were excluded from further analysis 
because they did not have high factor loadings. 

5.4.1 .3 Functions with medium range availability but high percentage of non-users 

On this note of availability, several functions were made available but were least 

used by a larger proportion of students - in particular, viewing slides or pictures on the 
course Web site (Q4, 23.2%), viewing multi-media presentation/lecture notes (Q3, 
17.6%), and communicating with content experts (QIO, 16.2%). Communicating with 
other students, via e-mail/listserv/hypermail/Chat rooms, etc. to discuss the ideas and 
concepts taught in the course (Q8) was used by only 38.7% of respondents. Interestingly, 
the data revealed that the function of communicating with course instructors and teaching 

’ Only missing value in all responses for Time Spent Category 




BEST COPY AVAILABLE 



assistants was perceived as available by 96.5% of students but had a relatively large 
percentage of non-users (43.7%), even though it ranked fourth in terms of high usage by 
users. 

5.4.2 Student Reported Time on Task 

All respondents were requested to report the amount of time each spent on a given 
function on the Web site in a typical week during the semester. Results pertaining to the 
INTERACT index are shown in Table 1 1 for each function. 
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Table 11 

Descriptive Statistics of Amount of Time on Task in Hours on a Typical Week 
During the Semester in Rank Order by Mean Time Spent 



INTERACT 




T»» • *11 

M = m mms 




Ql. Searching the WWW to access reference 

materials and/or conduct research for the course. 


133 


1.06 = 64 


1.07 


Q2. Downloading/printing information 
from the course Web site. 


137 


0.68=41 


0.75 


Q3. Viewing a multi-media presentation/lecture 
notes on the course Web site. 


116 


0.63 = 38 


0.90 


Q5. Picking up or completing assignments 
on the course Web site. 


127 


0.57 = 34 


0.67 


Q8. Communicating with other students, via 

e-mail/listserv/hypermail/Chat, etc. to discuss 
the ideas and concepts taught in the course. 


127 


0.57 = 34 


1.05 


Q9. Communicating with the course 

instructor or teaching assistant(s), via e-mail, 
to make a suggestion/complain/ask questions 
about the course. 


136 


0.40 = 24 


0.64 


QIO. Communicating with content experts 

outside your college or University via e-mail, 
or through the course Web site via hypermail 
or Chat. 


118 


0.19 = 12 


0.62 


INTERACT Mean Index Score 


142 


0.45 


0.48 



These results exclude non-users from the calculations based on the understanding 
that the means reflect average time spent on a given function by users of that function. 
Including non-users underestimates the mean time spent by users. 



Calculations exclude respondents who stated function is NA therefore this does not reflect N— 142 
' ' Minutes rounded off 




•i. 

X 
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As Table 1 1 shows on average, the longest time spent as slightly over an hour (64 
minutes) on searching the WWW to access reference materials and/or conduct research 
for the course (Ql). This was followed by downloading and/or printing information from 
the course Web site (Q2) that is approximately 41 minutes per week. On average per 
week, students spent at least 38 minutes viewing a multi-media presentation (Q3). They 
also spent approximately 34 minutes picking up and completing assignments (Q5) and 
communicating with other students via email, listserv, hypermail, or chat rooms, etc. to 
discuss the ideas and concepts taught in the course (Q8). 

Interestingly, while this result is mostly consistent with functions with the highest 
percentages of usage reported above in section 5.4.1, there appears to be some distinction 
between items that have a large number of users and items that students spend more time 
on. For example, while more respondents e-mail their instructors and their teaching 
assistants than those who e-mail their peers to discuss supposedly “academic matters”, 
the time spent on the latter function is much more. Similarly, while less than half the 
respondents view multimedia presentations and class lectures on the course Web site, 
those who do, spend a greater amount of time on this function compared to others. 

By contrast, students spent little time communicating with content experts outside 
their college or University via e-mail or through the course Web site, through hypermail 
or chat rooms (QIO). This result is interpreted with caution given that most of the Web 
sites did not highlight this function to any great extent, and for several classes, they were 
not available on many of the course Web sites. The findings are consistent with the low 
number of respondents (n) and percentages of users reported in Table 10 and discussed 
earlier. 
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Table 1 1 also displays time spent on each function in rank order on the 
INTERACT index. The difference between the highest mean score (Ql) and the lowest 
mean score (QIO) shows a difference of 0.87, which, in practical terms, means almost 
one hour (52 minutes). In addition, the difference in mean scores between searching the 
WWW (Ql) and the second highest function of downloading and printing (Q2) is 0.38. 
This translates into a difference of slightly under a half-hour (23 minutes). So while 
students spend approximately 41 minutes on average downloading and printing 
information from the course Web site, they spend an additional 23 minutes conducting 
research that may be accessible via the WWW that may or may not be readily available 
on their course Web sites. This result may have implications for students' use of time in 
terms of setting priorities for studying and also for printing costs when using the WWW. 
Students, having constraints on their time, may weigh its usefulness in relation to other 
means of conducting research. It also has implications for a Syracuse University policy 
imposing a 500-page per year no charge quota limit placed in effect as of January 2000. 
This is further discussed in chapter six. 

Students were also requested to indicate other functions they engaged in that 
were not included in the survey. Only 10% of the sample responded (n= 14). Results 
indicated that students spent more than 4 hours on average in a typical week on non- 
academic functions such as job hunting, searching for non-course related information and 
conversing with friends. They also reported functions already included in the survey such 
as obtaining their assignment/exam grades off the Web site, obtaining feedback, reading 
class armouncements, doing practice problems and downloading assignments. Other 
functions reported included looking up syllabi, or looking for extra credit opportunities. 
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and so on. These answers were not included in the statistical computations and are only 
reported in this section. 

It is worth noting that initially there appeared to be some inconsistency regarding 
how the communication function on the course Web site was used. Further analysis 
revealed that the purpose and type of communication is an important conceptually 
discriminating, factor. Results showed that, on academic related communications, there 
was a higher percentage of users communicating with instructors and teaching assistants 
(Q9, 24 mins.) than students amongst each other (Q8, 34 mins.). However, users spent 
more time communicating with each other than with instructors and teaching assistants. 
Meanwhile, the function of communicating with content experts outside their college or 
university was hardly used (QIO, 12 mins.). 

In summary, it is possible to conclude from the above findings that course Web 
sites were most frequently used by students for research purposes. This research might 
have included conducting searches or accessing material already compiled on the course 
Web site. This was followed by other interactive functions with course Web site material 
such as downloading/printing information, viewing multi-media presentations and class 
notes, picking up and completing assignments. Students also engaged in various 
communication activities, communicating each other and also with their instructors and 
teaching assistants, but spending more time communicating with each other. 





5.5 Question # 2 



What was the quality of the course Web sites according to students? 

This section presents and discusses findings regarding students' judgments of the 
instmctional quality of their course Web sites. In order to judge quality, an Instructional 
Quality category (Questions 15-32) was developed in the student Web site perception 
survey (SWPS). The Instructional Quality dimension was represented by 18 items (Alpha 
= 0.90) that attempted to measure the presence of Gagnes nine events in a Web-based 
context using a 5-point Likert-type agreement scale (0-4, where 0=No Basis for 
Judgment/Not Applicable and 4=Strongly Agree). Respondents were provided with a 
Not Applicable/No Basis for Judgment option to address non-availability of any given 
function or element on their respective course Web sites and these responses were not 
included in mean computations for each function. The rationale is that what is of 
importance here is the perceived quality of these various functions therefore, respondents 
who did not have an opinion: did not make a judgment quality. Therefore, only responses 
from students who actually formed some opinion about the quality of the Web sites were 
included in mean computations. Two constmcts were confirmed by factor analysis: 
APPEAL and PURPOSE and were used to create indexes for Instructional Quality ( Q) 
(see section 3.6.2 for index constmction). Table 12 and 4.5 respectively provide 
descriptive statistics for each item and index scores. 
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Table 12 



Descriptive Statistics on Students' Level Agreement with the Quality of 
Instructional for APPEAL Index in Rank Order by Item Means 
[Scale 0-4 (4=Strongly Agree)] 



APPEAL 


n'^ 


M 




a) Motivation and Web site Appeal (APPEAL) 








Q19. The Web site was easy to read. 


131 


3.16 


0.62 


Q15. The use of graphics, sounds, pictures, animations made 


91 


3.10 


0.58 


the Web site material more appealing to you visually. 








Q16. The combined use of text, graphics and/or sound to 


100 


3.08 


0.69 



represent the information in different ways enabled you to 
better understand the material. 



Q27. The introductions to Web segments were well-stated in terms of 


115 


2.96 


0.64 


clarity and purpose 








Q18. The colors used were in good taste and well coordinated 


116 


2.94 


0.62 


APPEAL Index Score 


132 


2.37 


0.37 



Results in Table 12 indicate that the highest rating of quality was the readability 
of the materials on the Web sites (Q19). This was closely followed by the use of 
appealing visual display (Q15). However, the lowest ratings were for the use of color in 
good taste (Q18). The introductions to Web segments were well-stated in terms of clarity 
and purpose (Q27). Perhaps better design of these features and functions may increase 
the time spent overall. The overall index score for APPEAL was slightly above average 
(M=2.37) and variation among groups was very small (SD=0.37). 



Calculations exclude respondents who stated function or characteristic is NA 



O 
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Table 13 

Descriptive Statistics on Students' Level Agreement with the Quality of 
Instructional for PURPOSE Index in Rank Order by Item Means 
[Scale 0-4 (4=Strongly Agree)] 



PURPOSE 


n’^ 


M 


SD 


b) Clarity of Purpose, Objectives, and Organization 








Q20. The material on the Web site was important for class. 


128 


3.41 


0.66 


Q22. The specific tasks for every assignment were specifically stated. 


124 


3.12 


0.77 


Q25. New material to be learned was well organized on the Web site 


93 


3.02 


0.71 


Q23. For each new section of the Web site, its connection to the 


120 


3.00 


0.77 


course objectives was made clear to you. 








PURPOSE Index Score 


132 


2.25 


0.41 



Results in Table 13 indicate that the highest rating of quality was the importance 
of the materials on the Web sites (Q20). This was closely followed by the specific 
statement of the required tasks for every assignment (Q22). However, the lowest ratings 
were for the connection made between each new section on the Web site and the course 
objectives (Q23). The overall index score for PURPOSE was above average (M=2.25) 
and variation among groups was very small (SD=0.41) but slightly larger than that for 
APPEAL indicating lesser agreement among groups that the course Web sites clearly 
stated the purpose and objectives of their use. In sum, the course Web sites were 
generally judged to be important and purposeful. New material was well organized into 
sections. The connection made between these sections to the course objectives, while 
rated highly appear in need of some improvement in comparison to other functions. 





121 



5.6 Question # 3 

How useful did students perceive their course Web sites? 

This section presents and discusses findings regarding students’ perceived 
usefulness of various functions of their course Web sites. In order to answer this question, 
a Usefulness dimension (Questions 33-47) was represented in SWPS using a 6-point 
Likert scale (NA, 0-4, where NA=Not Available/No Basis for Judgment and 4=Highly 
Useful). This category consisted of items that measured the extent to which students 
perceived certain functions as useful (Alpha=0.89). Respondents were provided with the 
option of stating that the function was Not Available on their course Web site or that there 
was No Basis for Judgment. The theoretical basis for these items was derived from 
Gagne’s nine events of instruction. However it is important to note that while these 
functions are conceptually related to the functions included in the Instructional Quality 
category mentioned above, they are nonetheless different primarily in the context and 
wording of the questionnaire items and in the factors that were confirmed for this index 
when factor analysis was computed. Tables 14 and 15 below present results for two 
indexes created to represent Usefulness^^ on two constructs PREPARE and GUIDE (see 
section 3.6.2 for index construction). 

Table 14 below provides means and standard deviations for all items. As can be 
seen from the results in Table 14, students reported that the use of pictures, tables, 
diagrams, etc. to recall or present new information was the most useful function of the 



Calculations exclude respondents who stated function or characteristic is NA 

ATTITUDE, a third construct of Usefulness is discussed later in response to research question # 4 in this 
study. 
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Web site (Q44). While all fimctions were perceived as useful as indicated by the lowest 
mean score (M=2.31, SD=1.03), the least useful function was providing links to review 
pre-requisite material to help recall (Q35). 

Table 14 

Descriptive Statistics of Students' Perception Of Usefulness for PREPARE Index in 
Rank Order by Mean [Scale = NA-0-4 (4=Highly Useful)] 



PREPARE 


D 


M 


SD 


Q44. The use of pictures, tables, diagrams, etc. to recall 


78 


2.79 


1.00 


or present new information. 

Q46. Overall, I have found the course Web site 


132 


2.63 


1.13 


Q45. The additional links and information on the Web for 


96 


2.43 


1.16 


further study. 

Q35. The links to review/prerequisite material to help recall. 


91 


2.31 


1.03 


PREPARE Index Score 


132 


1.87 


0.74 



Overall, results in Table 14 show low mean index score for PREPARE (n=132, 
M=1.87, SD=0.74). These functions represent Gagne’s events that state that students 
should be provided with the opportunity to recall prior learned knowledge to associate it 
with the new knowledge and also to be provided with the opportunity for further 
enrichment and remediation to facilitate transfer of this new knowledge. The capabilities 
of the WWW are to support visual display of information represented in the use of tables, 
diagrams, and other graphics. However, this low mean index score indicates that the 
Web sites need improvement particularly in the clarification of how the external Web site 
links relate to course objectives. 
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Table 15 

Descriptive Statistics of Students' Perception Of Usefulness for GUIDE Index in 
Rank Order by Mean [Scale = NA-0-4 (4=Highly Useful)] 



GUIDE 


n'^ 


M 




Q40. The opportunity to ask questions online 
(e-mail, listservs, hypermail, etc.). 


112 


2.74 


1.00 


Q41. Posting graded assignments/Home Work on the Web. 


64 


2.66 


1.30 


Q43. Online practice assignments (e.g. simulations, problems). 


47 


2.64 


0.85 


Q39. Online feedback on various assignments 
from Instructor/Tas. 


73 


2.60 


1.00 


Q42. Situations, case studies, formulas, problems, 

simulations, etc. posted on the course Web site to 
improve learning. 


65 


2.52 


0.99 


Q38. Online Self-Assessment Quizzes (where the answers 
are provided). 


49 


2.51 


1.23 


Q37. Instructions on how to navigate the Web site, 

download software, get an account, subscribe to a listserv. 


100 


2.30 


1.10 


GUIDE Index Score 


131 


2.20 


0.71 



Table 15 shows means and standard deviations on perceived usefulness of 
functions that pertain to the heart of the instruction as represented by Gagne’s events 
which state that performance should be elicited, opportunities for practice should be 
provided, and students should receive feedback on their performance as further guidance. 
These events are represented by the items listed in Table 15. Results show an overall 
higher index mean score (n=131, M=2.20, SD=0.71) compared to the one reported in 
Table 14. 

Calculations exclude NA responses 
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Respondents found that the opportunity to ask questions online was the most 
useful function and the posting of feedback on graded assignments as the next most 
useful function. This is closely followed by online practice assignments. It is evident 
that students appreciate the prompt feedback capabilities of electronic communication 
that is further enhanced by the visual capabilities of the WWW. These results provide 
evidence of the importance of feedback to clarify students understanding of new 
knowledge through one to one e-mail or from instructor to group communications by e- 
mail or through visual display of model answers, or automated feedback. 

In summary, all functions on the course Web sites are appreciated by students and 
perceived as useful, but to different extents. While, overall, students perceived their 
course Web sites as useful, there were large proportions of students who did not perceive 
most of these functions listed above as available. These large proportions of responses 
reveal a shortcoming in the optimal use of the WWW of equal importance to perceived 
usefulness. The above results point out various shortcomings in the current instructional 
design of these sampled Web sites and point to practical implications for optimal 
integration of the WWW to be discussed in chapter six. 




141 



125 



5.7 Question # 4 

What demographics and characteristics influenced students’ use and 
perceptions of quality and usefulness of course Web sites? 

As stated earlier in section 3.2, the purpose of this question was to explore 
possible influential factors on time spent on course Web sites and students’ perceptions 
of their course Web sites. While question #5 summarizes narrative data, this question 
explores statistical relationships among the following respondent demographics and 
characteristics based on the following assumptions: 

□ Residential Distance - students living off campus outside the 1-5 mile walking 
distance range were expected to give lower ratings on the usefulness of the course 
Web site than their counterparts living within the 1-5 miles range. Similarly, SU 
South campus students were expected to rate the course Web sites less useful and 
spend less time on the Internet than their SU North Campus counterparts. The 
rationale for both expectations is that living at a distance may require more support of 
learning given the inconvenience of driving to campus to meet with instructors and 
teaching assistants. This is based on the narrative data where SU South Campus 
students reported that they had no access to the Internet from their campus dorms. 
Second, given the academically-related inconveniences associated with living farther 
away from campus, it was expected that there would be more reliance on the course 
Web sites. Since these Web sites were not designed to fully support a distance 
learner, it was expected that students would experience some fiiastration. Thus, a 
likelihood of a negative correlation between distance and usefulness and between 
proximity and usefulness was anticipated. 
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□ American and International Students - International students were expected to 
spend more time on the course Web sites as indicated by the INTERACT index than 
their American counterparts, particularly since some international students might find 
it easier to communicate with their course instructors and teaching assistants than 
having face-to-face discussions which require more sophisticated verbal and non- 
verbal communication skills that differ from culture to culture. 

□ Type of Internet Connection - Two groups were compared on this characteristic, 
those who have an Ethernet connection and those who have dial-in only connections. 
Given that the dial-in only group might have been frustrated by slow connections, the 
students with Ethernet connections were expected to find the Web sites more useful 
since they could more easily log onto the Internet. 

□ Modem Speeds - Students with higher modem speeds were expected to spend more 
time on the course Web sites and to perceive them as more useful than their 
counterparts with slow modems, who might have been more easily frustrated by 
interruptions and busy line signals. 

□ Years of Accessing the WWW - Students with more years of accessing the WWW 
were expected to find the course Web site less useful than their counterparts since 
they might have been more capable of conducting their own research independent of 
what was made available through the course Web sites. Given that these Web sites 
were overall not very developed, it is likely that the more experienced students found 
these Web sites too basic for their more sophisticated expectations of the WWW. 

□ Courses - Differences were expected between courses and perceived Time Spent (T), 
Instructional Quality (Q), and Usefulness (U) given the variation in instructional 
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quality identified by WIDEC scores. 

□ Skills and Attitude - Students’ level of Internet skills and attitudes toward the course 
Web sites were expected to influence perceptions of T, Q, and U. A strong positive 
correlation was expected between SKILLS and T, Q, and U and between ATTITUDE 
and T, Q, and U. The rationale was that students with higher Internet skills were 
expected to find the course Web sites less useful as they can manage on their own and 
would spend less time on the course Web sites. Similarly, students with a positive 
attitude toward the course Web site were expected to spend more time using it and 
find it more useful than those who do not like the course Web site. 

In order to test these assumptions, an intercorrelation test was conducted for all 
demographics and characteristics including the indexes representing Time Spent, 
Instructional Quality, and Usefulness. Results are summarized in Table 16 below. 




144 



LEAVE PAGE BLANK FOR TABLE 16 




129 



Summary for only significant results of the exploratory analysis follows: 

Overall, the strength of the correlations was within the low to medium range. 

Examination of the stronger correlations reveals two types, ones that were expected to 
correlate given the context of the study and others that provided support for some of the 
above made assumptions. Stronger correlations overall were found between age and 
academic class (r=0.57), residential proximity and age (r=0.42) and residential proximity 
and class (r=0.64) where undergraduate students have traditionally lived on or very close 
to campus. A positive correlation was found between years of accessing the WWW and 
perceived Internet skills (r=0.29). There were no correlations found between sex, modem 
speed, and type of Internet connection and any of the study variables, 
a Residential Distance 

As expected, relatively strong correlations were found between residential 
distance in miles for off-campus students with students’ perception of usefulness indexes 
(r = -0.27 for GUIDE and r = -0.39 for PREPARE). Apparently, the negative strong 
correlation may have implications for students’ dependence on the course Web site the 
farther away they live from campus. These students may depend on the course W eb site 
to obtain material and communicate with instructors and peers more than their 
counterparts and therefore they are probably more critical of the usefulness of the course 
Web site. Descriptive statistics showed that the farther the distance from campus, the 
lower the perceived usefulness on the PREPARE index. An ANOVA test of variation 
among means also confirmed variations among residential distance of off-campus 
students (n=65, F=2.35, p<0.05) again specific to the PREPARE index comprising of 
functions related to recall, enrichment, and remediation. The Tukey test did not explain 
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this significance in findings. No significance was found among SU North campus and 
South campus students contrary to above noted expectations based on contextual factors. 

□ American and International Students 

Other interesting findings include a positive correlation between perceived 
Internet skills and the INTERACT index. Non-American students had more years of 
access to the WWW than their American counterparts as indicated by the negative 
correlation (-0.22) and more interaction on the Time Spent Index (r = -0.18) confirming 
the author’s assumption that there would be differences found on this index, albeit a 
very low correlation. 

□ Courses 

Results for ANOVA were significant for courses and students’ judgments of 
quality particularly in relation to functions relating t the PURPOSE index (df=8, F=4.33, 
p< 0.0001). A Tukey test showed specific variations between means for the HSTIOO 
course and three other courses: BIO400, SPP500, and FOR300. These results are further 
explained by narrative data where students in the HSTlOl course reported that they were 
obtaining everything they needed in class and were not sure about the purpose of the 
course Web site overall. HSTIOO received the lowest mean score amongst the group on 
the PURPOSE index (n=l 1, M=1 .8, ^=0.50). By contrast, the BI0400 course received 
the highest mean in the group on the PURPOSE index (n=28, M=2.50, SD=0.31). The 
narrative data indicated that the BI0400 course Web site was perceived as essential to 
class, as indicated by instructors’ emphasis on use by making some material only 
available on the course Web site. The FOR300 (n=21, M=2.4, SD =0.33) and the SPP500 
(n=6, M=2.5, SD=0.35) were also in a similar situation as the BI0400 as indicated by 
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statistics and also by student narratives. In addition, these courses had the highest mean 
scores among all nine courses. It appears that the course ranking on the WIDEC scale 
was not so consistent with students’ perceptions of the quality of the course Web sites 
specific to the PURPOSE index. Specifically, HSTIOO which ranked high on WIDEC 
scale received the lowest mean scores respectively on the Quality index. It appears that 
students’ judgment of quality, particularly in relation to the PURPOSE index, is 
influenced by required use and instructor support of the course Web sites. This is further 
discussed in chapter six. 
a Attitude 

Keeping in mind, the low reliability of the ATTITUDE index (Alpha=0.67), 
several strong positive correlations were found with PURPOSE (r=0.44), 
APPEAL(r=0.40), GUIDE (r=0.32) and PREPARE (r=0.52). Interestingly, this 
relatively high correlation between PURPOSE and ATTUTUDE further supports the 
above findings that students’ judgment of quality may be influenced by students’ general 
attitude toward the purpose and required use of the Web site. The more students’ are 
encouraged to use the Web site and perceive it as useful, the more likely they highly 
judge the quality of the Web site, 
a Other Related Findings 

GPA correlated negatively with years of accessing the WWW (r=-0.28) and with 
residential proximity (r=-26) but positively with students’ nationality, where international 
students had higher GPAs (r=0.33). A simple regression test revealed some variation but 
no linear correlation between motivation functions and Web site appeal (APPEAL) and 
student GPA (n=l 13, df=l, F=4.40, p<0.05, R-squared=0.04). 
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Intercorrelations among the indexes resulted in PREPARE strongly correlating 
with the other Usefulness indicator APPEAL (r=0.52) and also with one of the 
Instructional Quality indicators GUIDE (r=0.52). This may indicate, perhaps, that 
students' perception of the usefulness of functions that enabled them to prepare for class 
or to seek remediation and enrichment (PREPARE) are influenced, and vice versa, by 
their overall perception of the quality of the Web site functions that guide their learning 
(GUIDE). 

In sum, this concludes the exploratory analysis for factors influencing students’ 
perceptions of Time Spent, Web site Usefulness, and judgments of Instructional Quality. 
Results are not conclusive given the small sample size of the study, low correlations, and 
the lack of further detailed explanatory narrative data related to the above. Further 
research possibilities are suggested in chapter six. 

5.8 Question # 5 

What were some student reported barriers to and facilitators of use of 
the course Web sites and overall perceived impact of the WWW on 
students’ learning experiences? 

This section reports students' open-ended comments regarding their perceptions of 
their course Web sites that the closed items on the questionnaire might not have 
included. Interestingly, the comments seem to support the above findings and provide 
further explanation that is discussed in chapter six. 

5.8.1 Students' Perceptions of Facilitators to Successful Use of the Course Web Site 

Approximately 322 statements in total were provided by 138 respondents in 
response to Q12 in SWPS on the three greatest facilitators to successful use of the course 
Website. Overall, the three most helpful aspects provided by the course Web sites were 




133 



Guidance, Content, and Availability of Materials. While the results when computed in 
percentages did not provide very high figures (highest was only 9.4%), the following is a 
comparative analysis within those small percentages generated. Thus, in terms of 
frequency of reporting. Guidance and Content ranked as the two highest factors at 9.4% 
each. Availability of Materials ranked second at 9.1%. Access ranked third at 7.9%, and 
Ease of Communication ranked fourth at 7.2 %. Table 17 further illustrates this. 

Table 17 

Summary of Overall Most Helpful Factors for Total Sample* 



Most Helpful 


Second Most 


Third Most 


Fourth Most 




Helpful 


Helpful 


Helpful 


Guidance 


Availability of 


Access 


Ease of 




Material 




Communication 


9.4% 


9.1 % 


7.9 % 


7.2 % 


Content 








9.4 % 








Other 


% 


Web site Organization 


6.9% 


Remediation 


5.7% 


Currency of Aimouncements 


4.7% 


Orientation to Web site 


3 % 


Web site Design 


2.9% 


Enrichment and Professor Endorsement 


2.5% 


Motivation and WWW Features 


1.9% 


Speed and Ease of Use 


1.6% 


Navigation and Clarity of Information 


1.3% 


Time, Relevance of Information, and Printability of pages 


0.9% 


Convenience, Ease of Downloading, and Ease of Printing 


0.6% 


Appealing Web pages. Assessment, Class Assistance, Instructions 
Learning Style, Downloadability, Quality of Content, and 


0.3% 


Readability 









* Figures include rounding off error 



Guidance included statements related to providing students with directions, 
detailed written instruction on how to use the Web site. It also entailed statements about 
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what to do or know and detailed explanations. Guidance was perceived online in the 
form of the syllabus, professor/TA assistance via email, providing objectives for each 
chapter, relevant links, and pictures. It also included use of the Web site during class 
time. 

Content categorized statements that referred to the selection of Web sites that the 
faculty provided links to. It also included online presentation of course outlines, access 
to speaker notes, dictionary, and quantity and quality of information, hyperlinks to 
specific topics of relevance, slide files, links to research papers, and most of all class 
notes. 

Availability of Material is very similar to Access but was more specific to 
accessibility of specific information and content that is needed for class. It also included 
posting assignments online, and the ability to retrieve material or work that was missed 
due to absence. Some students mentioned that, for example, when they missed a slide in 
class, they could always go back to the Web site for the complete collection. In addition 
to slides, class notes, and hand-outs were also perceived as readily available and made 
accessible by the Web site. 

Access referred to statements about access to grades, having one's own computer, 
easy access to the Web site, the existence of a course Web site, and access to the site 
from any computer with a network connection. 

Ease of Communication referred to ease of contact with TAs, professors, and 
other students. This was particularly true with being able to contact the professor at any 
time. The availability of a biographic page for the Immunbiology course, or TA and 
classmates' e-mail addresses, was appreciated. Use of forms to submit to the faculty, be 




15 1 



135 



1 

it weekly assignments, quizzes, or other was also considered helpful. This method of 
being able to send information to the faculty, such as various assignments was also 
considered a means of completing class responsibilities. 

Finally, in addition to the above mentioned four most helpful factors, other issues 
emerged as also useful and helpful, including organization and design of course Web 
sites, remediation functions, and currency of announcements. Again, these issues seemed 
to emerge from across all nine courses in varying levels of importance as measured by 
percentage of occurrence. 

5.8.2 Students' Perceptions of Barriers to Use of the Course Web Sites 

Approximately 277 statements in total were provided by 140 respondents in 
response to Q13 in SWPS on the three greatest barriers to successful use of the course 
Web sites. Several factors emerged as barriers to use of the course Web site in response 
to Q13 in SWPS. Consistently throughout the data, very similar issues emerged as 
barriers in every course. This greatly facilitated the categorization of the data. Each 
category is described below and examples are provided. To begin with. Table 18 reports 
a summary of findings: 
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Table 18 

Summary of Overall Barriers for Total Sample * 



First Barrier 


Second 

Barrier 


Third 

Barrier 


Fourth 

Barrier 


Other Barriers 


Access 


Lack of Skills 


Motivation 


Time 


Organization 


5% 










Design 


5% 


17% 


16% 


13% 


12% 


Relevance 


4% 










Printing problems 


4% 










Lack of 


4% 










Information 












Connection 


4% 










Computer 


4% 










Problems 












Currency 


4% 










Content 


3% 










Speed 


3% 










Downloading 


2% 










Network 


1% 










Readability 


1% 



* Figures include rounding off error 



Access ranked as the highest perceived barrier across all groups (17%). 
Previously reported descriptive statistics showed 97.9 % access rate which was brought 
down by the fact that only 71% of the Phonation Disorders class had access. Access in 
the survey was defined as owning a computer or having regular access to one. However, 
in the open-ended question, respondents defined “access” to include waiting in line at 
campus computer clusters, and room-mates who did not always want to share their 
computers. Accessing certain links was included in this category. 

Lack of Skills closely ranked as the second perceived barrier across all groups 
(16%). Many students attributed their dissatisfaction to low self-ratings of computer 
literacy. Many reported that only by overcoming initial barriers, such as knowing how 
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to navigate a site, downloading, and staying focused, were they finally able to enjoy the 
experience and make use of it. 

Motivation, followed closely by Time ranked as the third and fourth barriers (13% 
and 12%) respectively. Many students reported that the desire to access the Web site was 
crucial; particularly when it was not obligatory or required by many of their instructors. 
This was particularly true for courses where the instructor did not provide any 
encouragement or make any demands on their students to use the Web site. Time was 
also perceived a barrier considering the trouble some students reported that they had to go 
through in order to access the site. Other related barriers that were mentioned included 
the time they spent waiting in lines at computer clusters, problem links of the Web sites 
that prevent them from accessing the references they desired, printing and downloading 
problems. Having access to this material through other means and the instructional worth 
of the content they have to access were two issues students reported to impact their 
perceived sense of time allocated for Web use. 

5.8.3 Impact of Course Web sites on Students’ Learning Experiences 

This section provides a summary of the findings of the content analysis performed 
on the open-ended question regarding students' perceived impact of the course Web sites, 
if any, on their learning experiences. The third open-ended question in SWPS (Q14) 
elicited students’ perception of the impact of the WWW on their learning experience. 
Overall, responses were highly favorable concerning the use of the technology with only 
some minor unfavorable comments and some indifference. 

Students reported that there were several advantages and a few disadvantages in using the 
course Web site and they are as follows: 
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> time-saving 

> facilitated preparation for class 

> increased understanding of class expectations, objectives, and facilitated learning 

> kept students "tuned in" to what was going on in class 

> provided enrichment in terms of content and access to a wide variety of resources 

> provided 24-hour access to content, faculty, teaching assistants, and class-mates 

> served as an easy and convenient communication tool 
^ value-added 

> paper wasting 

These findings are further discussed in chapter six in combination with findings from the above questions. 

> Time-saving 

Further elaboration of the above reveals that the time-saving function of the Web 
site meant that the course Web site made a difference in terms of lessening class time 
allocated to detailed note-taking, presentations, and slides and conducting research. The 
Immunobiology class, for example reported that posting class notes and slides increased 
their level of concentration in class by allowing them to pay more attention to concepts. 
It also allowed for more time to review the lecture-notes and to stay “in touch” with the 
material especially since they reported that there were many pictures that they had to 
recall and comment on. Similarly, the Phonation Disorders class reported that time was 
saved on note taking and on searching for readings. One student said, “I would fail 
because all my time would be spent looking up articles in books”. Similarly the students 
in the Physics class reported saving time on photocopying in the library. 

> Facilitated preparation for class 
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Students from the Soils class, for example, reported that the course Web site 
contained supplementary material that was not accessible from any other resource. They 
therefore believed that they would have been behind in lab materials without the Web. 
Meanwhile, the Physics class reported better access to content and instructional functions 
such as problem sets, answers to certain problems, and being able to review assignments 
or prepare for exams through the Web site. The Freshman Writing class reported that 
they were able to do their homework assignments on time and as scheduled because they 
were reminded of them through their course Web site. The Physics class also reported a 
similar situation. The Immunobiology class reported that the Web site further facilitated 
recall of graphs and diagrams and various domains of information. 

> Increased understanding of class expectations, objectives, and facilitated 
learning 

The Web site became a source of “ knowing what to expect ”, “knowing where to 
check up specific content ” and knowing “ what’s going on in class ”. Students from 
Immunobiology class for example, commented on the well-organized format of their 
course Web site that included class notes. Students reported that they were well laid out 
and easy to print and easier to study from because they know exactly what was taught. 
Others added that they could understand more of the concepts after viewing the 
information before class. In fact, knowing what to expect each day when going to class 
was considered another value-added feature. The Physics class for example, also 
commented on being better able to understand many of the problem sets due to receiving 
online help and studying tips for the exam. They added that solutions provided to 
homework assignments on the Web site were also considered helpful in studying for 
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exams. Consequently, some believed that the Web site had helped performance on tests 
and made the material more understandable. Similarly, providing quizzes ahead of time 
was also appreciated. The Immunobiology students reported that they found being able 
to prepare for the quiz ahead of time was most useful. In fact, one person reported that 
the availability of quizzes on the Web site was "the one thing that had made some impact 
on my educational experience if nothing else". Above all, some stated that easy access to 
old exams and material created a positive perception of studying. Printing out relevant 
exams at the time of need was perceived to facilitate studying. 

> Kept students "tuned in" to what was going on in class 

Many students appreciated easy access to announcements, updates on scheduled 
events and, above all, keeping abreast of what was going on in class when they had to 
miss class. The Immunobiology class, for example, perceived the Web site a good 
communication tool to obtaining information during absence or when there was a need to 
send assignments or messages over the Internet during vacations. Similarly, the 
Freshmen Writing students reported that the course Web site also updated absent students 
on class functions and therefore enabled them to keep up with class pace, while the 
Writing Studio class appreciated access to class assignments and updates on the syllabus 
and reported that scheduled events were easier to find out. In addition, students from the 
Immunbiology class considered the Web site a good source of information update that 
“lessens the need to have to get everything directly from the professor”. 
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> Provided enrichment in terms of content and access to a wide variety of 

resources 

The Web site also proved useful in that the students in Soils class reported having 
access to content-related information resources such as the “NRCS official Soils series”. 
Others reported constant use of the Internet to gather information for various reasons - 
educationally and socially. Some felt that they their educational experience has become 
better with its integration into their learning experiences. For example, some students 
explained how less laborious background information research has become. Not having 
access to the course Web site has a negative impact on research-focused courses because 
they are restricted to a narrower variety of resources. 

> Provided 24-hour access to content, faculty, Teaching Assistants, and class-mates 

Convenient access to material was reported by many, as one of the positive aspects 
about the course Web site. The Immunobiology students in particular, reported that they 
did not need to go to the library to get quizzes on reserve. In addition, picking up and 
dropping off assignments was made easier through online uploading and downloading. 
Access to material and to the syllabus when away from home was also a positive 
experience reported. This access facilitated completion of homework assignments that 
normally would have been difficult to obtain or turn in during absence. Students who 
attend lab session such as the Soils class added that another advantage of easy access to 
lab material is being able to print them out a few minutes before the lab. This is much 
more convenient than receiving them in class and holding on to them until the next lab 
session which is typically held once a week while the class is held three times a week. 
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> Served as an easy and convenient communication tool 

Others perceived access to the professor as another form of convenience and 
facilitation that prior to the Internet, they did not take advantage of Some students 
reported that they typically had little or no interaction with professors because they do not 
visit during office hours. Online communication with the instructor was mentioned 
several times in terms of having easy access to the professor to ask questions and 
converse rather than having to “track him/her down”. 

> Value-added 

In sum, many students considered the Web site as a convenient means of having 
access to content and therefore did not need to meet face-to-face with the TA as much, 
take detailed notes in class, or go to the library every day. Communication, a strong 
function of the Internet, served to increase the feeling of “cormectedness with the 
professor” especially when the Economics class students explained that they had to give 
feedback about the course once a week. Greater access to not only assignments but also 
to people and realizing the potential of electronic educational aids was a value-added in 
addition to conducting research, as reported by the Freshman Writing class. Another 
beneficial “side-effecf’ was reported by the Soils class mentioning that the course Web 
site facilitated easier use of the Web in other subjects, a value-added increase in 
knowledge and skills in using the Internet. They also reported that using the course Web 
site made the course much less time consuming in terms of logistics and preparation time. 

In terms of negative impact, students reported that printing documents was a 
waste of much paper. Some students reported using over 500 papers in one semester. 
This has implications for the 500-page quota limit imposed in Jan 2000 mentioned 
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earlier. Another negative impact reported is the possible loss of valued teaching styles 
such as lecturing and face-to-face communication. Several students reported no impact 
whatsoever because it was only one alternative resource amongst several and therefore 
there was no total reliance on the course Web site so its impact was hardly felt or 
conceptualized. In conclusion, the above findings have implications for instructional 
design theory and practice and motivational theory and adoption of innovations theory, 
in particular. These findings would be discussed in chapter six using these theoretical 
frameworks to ground the study findings. 

5.9 Chapter Summary 

This chapter has delineated the results of the study based on a survey of 142 
students enrolled in nine course that use a course Web site as an instructional support 
resource to classroom based teaching. The chapter presented results of selected course 
Web site functions pertaining to three main indexes: Time Spent (T), Instructional 
Quality (Q), and Usefulness (U). In addition narrative data were categorized into various 
themes related to barriers to and facilitators of use and overall impact of the WWW on 
students’ learning experiences. Results overall are favorable of WWW use particularly 
for functions that guide, motivate, prepare, and provide a purpose for students to use the 
course Web site. Students spent most of the time interacting with material and with 
instructors. However, the quality of interaction with materials could be more efficient 
and productive and interaction with experts could be increased. Summary of findings 
follows in chapter six. 
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CHAPTER SIX 

Discussion 

6.1 Introduction 

The study sought answers to the following questions: (1) How did students 
allocate their time using their course Web sites? (2) What was the quality of the course 
Web sites according to students? (3) How did students perceive the usefulness of their 
course Web sites? (4) What demographics and characteristics influenced students' use 
and perceptions of quality and usefulness of course Web sites? (5) What were some 
student reported barriers to and facilitators of use of the course Web sites and overall 
impact of the WWW on students' learning experiences? 

The purpose of this chapter is to discuss the findings of the study in light of 
relevant research findings and literature on optimal use of the WWW to support teaching 
and learning practices. The discussion focuses on methodological, theoretical, and 
practical implications relevant to the study. This chapter is organized around the 
following sections: 

□ Summary of Findings 

□ Theoretical Implications of Findings 

□ Practical Implications of Findings on Teaching and Learning Practices 

□ Methodological Implications 

□ Future Research 

□ Conclusions and Recommendations 
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A synthesis of findings is presented using a thematic organizational framework 
for discussion. The primary intended audience for this discussion is higher education 
faculty, students, administrators and instructional designers interested in improving 
teaching and learning practices by facilitating and supporting the integration of WWW 
technologies to support educational and training practices. 

6.2 Summary of Findings 

Study findings indicated an overall positive perception of the quality and 
usefulness of the course Web sites. In terms of time spent on given functions, on average 
per typical semester week, 64 minutes was spent on conducting searches on the WWW, 

40 minutes on downloading and printing material, and 34 minutes on communicating 
with faculty and teaching assistants. The highest ratings of instructional quality were on 
the visual appeal and readability of Web sites and the importance of the material on the 
Web site. The lowest ratings of instructional quality were on the clarity and purposeful 
introductions to each segment on the Web site, the clarity of the connection of each new 
section on the Web site with course objectives, and the general taste in color of pages. 

The highest ratings for perceived usefulness were on the use of visuals to recall or present 
new information and the opportunity to ask questions online. The lowest ratings were the 
use of links to review/recall prerequisite material and instruction on how to navigate the 
Web site. 

Students reported that the greatest barriers to use were access to computers and to 
Web site addresses (URLs), perceived inadequacy of their Internet skills, motivation to 
use the course Web site, and time constraints. The greatest facilitators of use were 
guidance, quality of content, availability of material, access to material, faculty, peers. 
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teaching assistants, and experts, and ease of communication. Overall impact of course 
Web site was its time saving qualities, 24-hour accessibility to resources, facilitating 
preparation for class, and increased understanding of class expectations and objectives. 
Finally, a negative relationship was found between residential distance from campus and 
perceived usefulness of course Web sites and a possible relationship between courses and 
students’ perceived instructional quality specifically on functions related to clarity of 
purpose and objectives. In addition, there appears to be a general lack of motivation to 
use the course Web sites, possibly due to their lack of mandatory use and what students 
reported as a lack of incentive to use them for specific course requirements. Table 19 
provides a summary table of findings. 
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Table 19 



Sum mary Table of Findings 



Dimension 


INDEX/Function - s | 


Time Spent 


Usage 


INTERACT 


Highest /Most 


Searching the WWW access 
reference materials and/or to conduct 
research for the course 


Searching the WWW access reference 
materials and/or to conduct research for 
the course 


Lowest/Least 


Communicating with other 
content experts 
Taking an online Quiz 
Creating a Web page 


Communicating with peers to discuss 
academically related matters 


Instructional 

Quality 


.APPEAL. . . 


r? PURPOSE 


Highest 

Quality 


Easy to read Web site 
Appealing visuals 


Importance of material on Web site 


Lowest Quality 


Clear and purposeful introductions to 
each segment 

Taste in color coordination 


Clarity of objectives for each new section 
of the Web site 


Usefulness 


PREPARE 


GUIDE 


Most Useful 


Use of visuals to recall or present new 
information 


Opportunir\- to ask questions online 

i 


Least Useful 


links to review/recall prerequisite 
material 


; Instruction on how to navigate the Web 
; site, download software, get an account, 
I subscribe to a listserv. 


Influential 

Factors 


Demographics 


Characteristics 




Non- American students had more years 
of WWW access 

Non-American students spent more time 
on the WWW 
(low correlation) 


The farther the residential distance the 
lower perceived usefulness 
There was a relationship between course 
and perceived quality (PURPOSE) 
Strong correlation between students' 
attitude and perceptions of quality and 
-Usefiilness (PREPARE) 


Satisfaction 


Greatest Barriers to Use 


Greatest Facilitators to Use i 




Access to computers and Web site links 
Lack of Skill 
Motivation 
Time Constraints 


Guidance 

: Availabiliu- of Material 
Access to material and people 
Ease of Communication 




Impact of W'WW ! 




Time-saving 

Facilitated preparation for class 

Increased understanding of class expectations and objectives 
Kept students "tuned in" to what is going on in class 

Provided enrichment in terms of content and access to a wide variety of resources 
Provided 24-hour access to faculty, Teaching Assistants, and class-mates 
Served as an easy and convenient communication tool 
Value-added 
Paper wasting 
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6.3 Theoretical Implications of Findings 

This section discusses the findings using the most applicable to findings from the 
seven principles of good teaching and learning practices (Chickering & Ehrmann, 1996) 
and the nine events of instruction (Gagne, Briggs, & Wager, 1992) as the theoretical 
framework for the discussion. Other theories and supporting discussions are drawn upon 
from related literature. 

6.3.1 Students' Perceptions of Web site Use 

6.3. 1.1 Good practice emphasizes time on task 

Findings on technology use are consistent with Chickering and Ehrmann's 
(1996) principle stated in the subheading of this section. On average, students spent 
approximately 64 minutes of a typical semester week searching the WWW to conduct 
research. This function is where students spent most of their time. Given this finding, 
two interpretations are considered here: the time spent searching the WWW may have 
been spent productively, or time was spent unproductively. Evidence to support the 
latter contention can be found in the narrative data, where Time and Lack of Skills 
emerged as two of the three main barriers to the use of the Web sites for all students. 
Time constraints were reported as not having enough time to "sit down and do it". 
Students complained about slow modems that take them longer time to download 
pictures and graphs. Others mentioned that some sites took too long to be displayed. 

Across all disciplines, students reported that they do not have certain skills. For 
example, the Freshmen Writing class reported: lack of familiarity with the WWW, not 
knowing how to navigate the Web site, and difficulties in retrieving information from 
the Web site. Students in the Soils class reported that the searchs were too broad and 
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that they were trying to figure out how search engines really worked or how to 
bookmark a site. Some students reported that they didn't know how to join a listserv. 
Students in the Immunobiology course reported that they did not have the skills to 
download information or "a great understanding with [of] computers". Students in the 
Economics class reported that they had trouble with the "graph tune up" program and 
did not know how to use it well. In sum, in every class, many students reported not 
having adequate skills that would allow them to use the course Web sites more 
efficiently. Finally, to compound the problem, many students reported that they had 
difficulty finding the course Web site! 

Of concern here to the instructional designer and perhaps the course instructor is 
optimizing the use of the medium to make efficient use of time and orienting students 
on using the course Web site. Also, the designer or instructor should give attention to 
designing user-fnendly Web sites that do not require students to download software, but 
are rather facilitative and supportive of their cognitive learning and information 
processing strategies. This could reduce the time spent on downloading software and 
printing text and pictures by increasing time spent on active engagement with the Web 
site. For example, El-Tigi and Branch (1997) suggested utilization of three popular 
Web design features - frames, tables, and maps - to organize Web sites to facilitate 
cognitive learning strategies described by West, Farmer, and Wolf (1994), in a Web- 
based context. These features could become organizational techniques that represent 
conceptual associations, optimizing information presentation to better represent the 
cognitive strategies discussed by West et al. (1994). For instance, frames utilize the 
technological capability of multiple screens, the user can scroll through one screen 
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while making associations with another. Image maps can be placed in strategic 
locations and can serve as navigational aids to the student. Tables can provide 
organizational structures that save on printing and downloading pages of text. Tables 
also may enhance learning by facilitating conceptual associations through their multi- 
dimensionality using vertical and horizontal associations between cells and also through 
hyperlinking. Should students need to print material, it is limited to a few pages of 
synthesized information. 

6.3.1 .2 Good practice develops reciprocity and cooperation among students 

The second important finding is that there is a clear indication that students spent a lot 
of time communicating with each other through email. While students were surveyed 
regarding communications specific to discussions of academic-related matters, they may 
also have used email simply to socialize. No further information is provided by the data 
on this matter. Assuming that students do communicate about the course, then these 
findings provide empirical evidence for Chickering and Ehrmann's (1996) principle 
previously stated as a subheading. Students learn through discussions, reflective writing, 
and making meaning out of what they are learning by relating it to previously acquired 
knowledge and skills. This learning experience can be facilitated and enhanced through 
e-mail communications (Anderson & Lee, 1995; Tao & Reinking, 1996). The WWW, in 
this instance, serves as both a communication tool and as a search tool that brings a large 
information database to the students' fingertips. 

In the narrative data. Ease of Communication was one of the three most helpful 
functions of the Web sites. Using forms to submit to the faculty, be they weekly 
assignments or quizzes, were also considered helpful. Having an email link on the course 
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Web site to faculty, teaching assistants, and even peers was considered an easy way to 
"get in touch" regarding course-related matters. A good example of this is the 
Immunobiology course Web site which had a student biography page that linked to their 
pictures, emails, and special interests - a feature that students reported facilitated 
communication. 

It is evident that the course Web site provided easy access to email addresses of 
faculty, teaching assistant and peers. A review of the literature on the effects of e-mail 
communications in various contexts showed overall positive impact. In a study 
conducted by Anderson and Lee (1995), use of e-mail in a reading education class 
showed that e-mail played a major role in building a sense of community, facilitated the 
sharing of ideas and resources, encouraged risk taking, helped students reflect on their 
learning, and encouraged cooperative learning. E-mail communications have also been 
found to give students a better sense of audience (Schwartz, 1990). In addition, 
increased student and instructor interaction patterns were reported (Dreher, 1984). In 
their review of research about email in education, Tao and Reinking (1996) cited 
favorable findings from several studies that took advantage of some features of email 
communication. For example, overcoming temporal and spatial distances (Schwartz, 
1990); and using email's relative anonymity and asynchronous features to encourage 
students to speak out (Fey, 1994; Anderson & Lee, 1995). The above explains the 
correlation found between international students and time on task on the interactive and 
communication functions. However, the negative aspects of email are also important to 
consider, but are not as relevant to this discussion. 
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6.3.2 Students' Perceptions of the Instructional Quality and Usefulness of 
Web site Functions 

Key findings from the study are indicated below using the most relevant of 
Chickering and Ehrmann's (1996) seven principles of good teaching and learning 
practices for undergraduate education as the framework for discussion in combination 
with Gagne’s nine events of instruction. In addition, relevant findings from the study 
serve to add to the limited empirical evidence regarding the relative merits of media 
selection models as pointed out by Reiser and Gagne (1983) in their review and 
evaluation of the learning effectiveness of the major features found in media selection 
models. The findings also serve to contribute to the empirical evidence for appropriate 
use of the Web as an instructional medium. Because the initial events of a lesson are 
often designed to re-arouse motivational states in the learner, the discussion will 
emphasize the importance and relevance of motivation theories to the application of Web 
design. Keller’s ARCS model and its application in a Web-based context will be an 
integral component of the discussion of the nine events. 

6.3.2. 1 Good practice encourages contacts between students and faculty 

An important finding was students' perceptions of the usefulness of 
communicating online. Electronic mail, computer conferencing, and the WWW have 
increased opportunities for students and faculty to interact and communicate much more 
efficiently than before. According to Chickering and Ehrmann, 1996, the nature of 
communication becomes more thoughtful, deliberate, convenient and safe. Moreover, the 
negative aspects of face-to face confrontation are lessened. This may be particularly 
relevant to international students. 
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6. 3. 2. 2 Gaining Attention - Motivation! 6 

Key findings from the study indicated that motivation in general is an issue that 
has emerged as important to this study’s findings. The Gaining Attention component of 
the nine events is very limited in its presentation of motivational elements and therefore 
the ARCS motivational design model (Keller, 1983, 1987a, 1987b, 1987c, 1988) is used 
here to frame and guide the discussion. The ARCS model has four general requirements 
that must be met in order to motivate people to learn: (a) Attention, (b) Relevance, (c) 
Confidence, and (d) Satisfaction. Attention is a prerequisite for learning. It directs a 
learner to the appropriate stimuli. Relevance is connecting instruction to important needs 
and motives. Confidence influences a learner’s persistence and accomplishment. 
Satisfaction entails natural and positive consequences and a sense of equity that are 
necessary elements to make learners feel good about learning results, so they will be 
motivated to learn. (Keller, 1987c, 1988). According to Keller (1983), one consequence 
of motivation is "to contribute to better learning" (p.388) and hence the quality of 
instruction. Another consequence is "the intensity of performance at a task" (p.388). 

Key findings in the study identified a general lack of motivation to use the course 
Web sites. Specifically, motivational barriers were reported by the History, Economics, 
Public Affairs, and Physics classes. For example, in the Physics class, students reported 
that they did not have the desire to check the Web site. The Freshmen Writing course 
reported that the site had a complex URL address. They also reported that their instructor 
did not mention it very often so they developed a general lack of interest in the Web site. 



One of the nine events “ Stimulate recall of prior learning “ is not separately discussed due to insufficient 
data. 
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perceiving there was no need to use it. Some did not like to read from the computer 
screen, as well. 

The History class reported that the URL address was also difficult to remember, 
and that they generally forgot about the existence of the Web site, especially since there 
was no explicit incentive to use it. Students from the Economics class reported that they 
did not have a need to use it, and that they had difficult remembering the address even 
when they had to submit a weekly online report. Likewise, both the Public Affairs and 
Writing Studio classes reported a general lack of incentive to use the course Web sites. 

It is important to point out that these findings were not surprising given that data from the 
faculty survey emphasized that instructors did not require mandatory use of the course 
Web sites for fear of some students not having regular access but the instructors did 
encourage its use. Nonetheless, besides teacher encouragement and mandatory use, 
course Web sites can be designed to be motivating. A discussion of the application of 
Keller’s ARCS model to a Web-based context seems appropriate at this point. 

Using a Web-based context. Small (1997), drawing upon Keller’s ARCS model as 
a theoretical foundation, identified and empirically validated four general categories of a 
motivational Web site: engaging, meaningful, organized, and enjoyable. Students’ 
attention may be gained by initial engagement, and this engagement is maintained by 
varying the look of activities at the site. Moreover, inquiry arousal sustains attention. 

This can be achieved by creating a problem situation that can be resolved by knowledge- 
seeking behavior. Problem solving warm-up activities using questioning techniques can 
stimulate curiosity and exploratory behavior. 
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In a Web-based environment, Small (1997) recommended that initial engagement 
must occur at the Web site starting point: the home page. She suggested the use of "eye- 
catching visuals, attractive screen layout, and humor [as], . . strategies that can grab 

the users’ interest and arouse his/her curiosity" (p. 7). Relevant to this discussion is an 
interesting finding from this study. Students ranked visual appeal as a mark of high 
course Web site quality. A good example of some of these strategies was found on the 
Immunbiology course Web site where students were invited to submit a paragraph about 
themselves and a digitized picture was taken of them and posted on the Web site. 
Students reported their appreciation of this in their narratives, indicating that in a class of 
89 students, getting to know each other through the course Web site was a great feature. 
Another example was provided by the History course where students could access a page 
containing photographs of a number of famous historical figures that they were asked to 
identify. Once they clicked on the picture, biographical data of the figure was provided. 
While the above is very feasible in a Web-based environment, the multi-media 
capabilities of the medium can further support such efforts through use of visual displays 
that engage students in mystery solving, offering answers through further exploration. In 
sum. Small (1997) recommended that relevance of a site must be established at the 
moment of access by providing a meaningful context to users by stating the purpose and 
importance of the Web site. This recommendation is supported by findings from this 
study where stating objectives also ranked highest in terms of quality of students’ 
perceptions of Web sites, which is discussed next. 
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6. 3. 2. 3 Informing the Learner of Objectives 

Key findings from the study indicated that for all events of instruction, the level 
of agreement regarding the quality of instruction delivered via the course Web sites was 
highest in terms of clearly stating objectives. Students’ highest ranking of quality, on 
one measure, was that the Web sites were easy to read, thereby adhering to a fundamental 
condition in presenting objectives. Another important finding is that the links to other 
sites were clearly related to course objectives and students highly rated experiencing no 
difficulty in following through links and making associations with original course 
objectives. The Web site objectives/purposes/expectations were stated in relation to 
overall course goals and objectives. A good example of this was found on both the 
Economics and the Soils class Web sites where course objectives were clearly stated in 
relation to the purpose of each major section on the course Web site. The Economics 
course Web site in particular clearly stated course objectives for every major section of 
the Web site. Students also appreciated this function in the narrative data. The findings 
support those of Schlopugh and Bhuripanyo (1998) reported earlier in section 2. 5. 2. 3 
where the clarity and importance of online documents in relation to course content 
increased the sense of connection with the course - considered a strength of the course 
Web site. 

The importance of stating purposes and objectives as reported by findings is 
particularly interesting since the hyperlinking feature of the Web has been known to lead 
to information overload and a sense of disorientation and loss of control, and cognitive 
overload as the two most commonly identified problems in navigating through 
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hypermedia (Jonassen & Grabinger, 1989; Park, 1991). This state of disorientation was 
described as follows: 

Many hypermedia systems consist of hundreds of nodes with a 
potentially confusing array of links connecting them. It is well 
documented that in such systems, users can easily become lost, not 
knowing where they came from, where they should go, or even how to 
exit the program they are in. Users are frustrated by this experience, 
frequently losing sight of their original purpose [italics added] in using the 
hypermedia. Often they give up without acquiring any information from 
the hypermedia. (Jonassen & Grabinger, 1989, p. 15) 

It is possible to predict that future research may find that overt and clear statement 
of objectives may not only play a large role in reducing students sense of loss and 
disorientation while navigating the Web, but actually may take on a very 
important role as a motivator for students to explore the Web as a guide-on-the- 
side. Hence, clearly stating objectives may have also increased students’ 
perceptions of the importance of the Web site. Both the statistical and the 
narrative analysis of the data identified the importance of a course Web site as a 
mark of quality on one indicator. The data also identified a need for specifying 
purposes and objectives for each section of the Web site so students have clarity 
in terms of expectations and how to go about navigating the site to accomplish a 
task at hand. A related finding that provides support to informing students of 
objectives and purposes of Web sites is Guidance. This is discussed in later on in 




section 6. 3. 2. 5. 
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Furthermore, setting educational goals, objectives, and purposes has been widely 
acknowledged in education research, particularly Bloom’s taxonomy of objectives 
(Bloom, Engelhart, Furst, Hill, & Krathwohl, 1956), which categorized objectives into a 
hierarchy of six classes (knowledge, comprehension, application, analysis, synthesis, and 
evaluation). The purpose of clearly stated objectives is to clearly define students’ 
understanding (Bloom et al., 1956) and thereby establish a relatively specific expectancy 
concerning the outcome of learning (Gagne, 1975). When students are informed of what 
they will engage in during their learning session, be it online or in class, and what they 
will be able to accomplish upon its termination, this facilitates in directing students' 
attention to material that will enable them to attain the specified expectancies. In an open 
environment such as the WWW, this might be crucial to identifying the scope and focus 
of each learning session so as to: (a) make better use of students time; (b) facilitate 
navigation, and (c) direct students attention to what is really important and relevant. 

So that while they “freely” explore the WWW, they are able to make better choices on 
navigation, on what to download and print, and on which cognitive strategies they need to 
draw upon to accomplish their tasks and purposes. 

Conceptually, it can be argued that there is a logical link between knowledge of 
objectives and perceived importance of a Web site leading to an increase in perceived 
relevance of the instructional functions on the Web site. Relevance of the task-at-hand to 
purposes, which is a necessary motivational requirement presented above in the ARCS 
model, is also central to creating meaningful Web-based learning environments. 
Relevance was identified as a ‘central concept’ to information science, a fundamental 
criterion for evaluating the effectiveness of information retrieval (Schamber, 1994 as 
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cited in Small, 1997). In sum, in a Web-based learning environment, given the inherent 
disorientation problems of the medium, overt and clear statements of expectations are 
crucial in serving not only as conceptual navigational aids but also as motivational aids 
and virtual guides for further student explorations of the WWW. 

6. 3.2.4 Presenting the Stimulus Materials 

This event, Bretz (1971 as cited in Reiser & Gagne, 1982) contended, is best 
delivered by a medium other than the classroom instructor. His rationale is that the 
instructor's time can be better spent on tasks requiring "a greater degree of human 
judgment and personal interaction, such as may be involved in diagnosing learner 
problems" (p. 508). Findings from this study support Bertz' (1971) contention in that 
students reported that having class notes posted on the course Web site freed them from 
the burden of note-taking during class time and allowed them to focus on interacting with 
the instructor at a higher level of conceptualization. 

Reiser and Gagne (1982) suggested that instructional stimuli should be similar to 
the stimuli involved in the performance to be learned. For example, when a learning task 
requires that visual features be distinguished, then visual media such as animation, 
graphics, and pictures should be employed. Similarly, auditory features should be used to 
present auditory stimuli. In this study, narrative data provided empirical evidence for 
students' great appreciation for the various multimedia capabilities employed to present 
information. For example, they appreciated the inclusion of slides and pictures for 
courses in which they were required to study hundreds of slides and pictures, as the case 
with the Soils and Immunobiology courses. In addition, narrative data also provided 
insights into students’ appreciation of the auditory representation of various sounds in the 
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Phonation Disorders class. Both the visual and auditory stimuli were made available 
online to students through their respective course Web sites. Thus, students reported they 
were able to practice numerous times through this medium what would not have been 
possible or as valuable in one lab session. Students noted that this also allowed them to 
make better use of their classroom interactions with the faculty knowing that the slides 
were available outside of class on a 24-hour basis. 

6. 3. 2. 5 Providing Learner Guidance 

Reiser and Gagne (1982) indicated that visual media is particularly useful for 
providing learner guidance. In particular, such media can "influence the semantic 
encoding process by helping learners establish mental images, which in turn are capable 
of enhancing retention (Gagne & White, 1978, p.508). The findings in this study also 
support this statement in that students had high expectations for the site in providing 
support and guidance. Evidence for this is found in the results where the most useful 
function on one indicator on the Usefulness dimension was the use of visuals to recall or 
present new information. As stated earlier, students appreciated Guidance. In this case, 
students expressed a need for more guidance to be incorporated in the design of the 
course Web sites. 

Guidance emerged in the narrative data as the most helpful factor in using the 
course Web sites. As stated earlier in chapter five. Guidance was described by students 
as providing directions, statements about what to do or know and detailed explanations. 
Guidance was also perceived online in the form of the syllabus, TA/Professor assistance 
via email, providing objectives for each chapter, relevant links, and pictures. It also 
included use of the Web site during class time as a means of integrating the information 
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on the Web site and giving it importance. For example, students in the Economics 
course appreciated this function by stating that clear written instructions on the course 
Web site and pages with important links were provided. The Immunobiology students 
informed us that the detailed explanations, descriptions, and instruction on the course 
Web site were appreciated guidance. The Public Affairs students appreciated the helpful 
guidelines and the links to relevant information. The Soils class referred to Guidance as 
knowing what and when to access specific information for studying pictures necessary to 
prepare for the weekly lab class. The Phonation Disorders class appreciated receiving 
guidance on the lab material. All classes considered having the course syllabus online 
important, as well. 

6. 3.2. 6 Eliciting Performance 

Eliciting performance is discussed in many media selection models (Gagne & 
Briggs, 1979; Gropper, 1976; Romiszowski, 1974; Tosti & Ball, 1969, as cited in Reiser 
& Gagne, 1982). Several of these models indicated that responses required of students 
should be categorized by type (e.g. overt, motor, verbal). Media are then selected 
according to how best they are able to elicit these particular type of responses. A 
limitation of this study is the minimal data available on the event dealing with eliciting 
responses from students. Results of the Web site course analysis performed by the author 
showed minimal use of this event in many of the course Web sites sampled. Further 
research in this area may lead to more insights. 

6. 3. 2. 6 Providing Feedback 

According to Reiser and Gagne (1982), there are several perspectives regarding 
media selection for this event. One view suggests that any medium is appropriate (Bretz, 
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1971) because any medium can provide knowledge of results. Another view indicates 
that the media used to provide feedback should depend on the type of learning outcome 
expected (Reiser & Gagne, 1982). For example, when learners are attempting to acquire 
intellectual skills it is desirable to provide them with correct and precise feedback. 
According to the authors, this type of feedback is well accommodated by interactive 
media such as computers. While a third view proposes that learner characteristics be 
considered when presenting feedback (Gropper, 1976). For example, error-prone 
learners may benefit from media that provide remedial branches. Findings from the study 
indicate that students greatly appreciated feedback on their problem sets and self- 
assessment quizzes provided online. Functions related to receiving feedback ranked 
highest on one of the Usefulness indicators presented in chapter five. More explicitly, it 
was the opportunity to ask questions, access graded assignments and homework answers, 
and receive online feedback on the above that ranked as most useful on the list of 
functions. These findings are worth comparing to findings from a study conducted by 
Kara and Kling (2000) who identified student fhistrations regarding pedagogical issues, 
including uncertainty of instructor expectations, lack of prompt feedback, minimal 
feedback, and ambiguity of instructions on the Web. 

6. 3. 2. 7 Assessing Performance 

In this study, online assessment was the weakest feature on all Web sites, perhaps 
due to the technical complexity, privacy, and security issues pertaining to posting tests 
online for public access. Therefore online assessment, did not have high factor loadings 
and had to be excluded from subsequent analysis. Nonetheless, because there were some 
examples of online practice quizzes and postings of old exams and answers online. 
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students were asked to comment on the usefulness of this event. Prior to its exclusion 
from the index, descriptive statistics showed that online self-assessment quizzes where 
the answers are provided were considered useful by students and ranked above average 
and considered more useful than (a) additional links; (b) online feedback from instructor; 
(c) using email to reflect on course work; and d) receiving instructions on how to 
navigate the Web site. Further research on the optimal use of the WWW to support 
assessment strategies is necessary. 

6. 3. 2. 8 Enhancing Retention and Transfer 

Reiser and Gagne (1982) stated that visual media can often be used effectively to 
enhance retention and transfer: "Visual media can help learners to acquire retainable 
visual images, which are later used as sources of cues for retrieval of what has been 
learned" (p. 5 10). Results supported the findings showing that the most useful functions 
on the Web site were the use of pictures, tables, diagrams, etc. to recall or present new 
information. The importance of this function was revealed by results that showed that 
students' judgment of the instructional quality of the Web sites was lowest for functions 
that facilitated their abilities to provide a rich environment to explore the topics covered 
in the course syllabi, and to make use of the information on the Web site by synthesizing 
it in various assignments such as the use of Web site links. This function appears to be 
weakest in the sample course Web sites. The data in this study were not sufficient for 
further analysis. Further research is needed to identify strategies and functions that the 
WWW can utilize to support this instructional event. 

6.4 Practical Implications 
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Three main dimensions were examined in this study, and practical implications 
are presented for each of the following: Use (Time Spent), Instructional Quality and 
Usefulness. Other related implications from the narrative data and the exploratory 
analysis are also included. 

6.4.1 Practical Implications for Web site Use 

The purpose of a course Web site is a main issue influencing use as indicated 
throughout the findings in this study from both the student and faculty surveys. Each 
course was designed by the faculty member with a different purpose in mind than the 
students, and this may have impacted the frequency of use and students’ perception of the 
importance of the course Web site. 

It appears that the more the course instructor relied on the course Web site as a 
resource, the more he/she encouraged students to use it and had higher expectations that 
the more students would use the site and have something to say about it in the narrative 
data. However, in all sampled courses, there was no obligatory use of the course Web 
sites, though some faculty made certain material available only through the course Web 
site. It is recommended that instructors think carefully about the purpose of the course 
Web site, to make serious use of it and to convey this message to their students. Putting 
material on the Web site and also handing it out in class serves only the few who miss 
class on a given day and sends mixed messages about the importance of the WWW. 
Students may assume that everything else on the Web site is also provided in class, as 
was the case with the History course in this study. As reported by the student who 
developed this site on the faculty survey, however, a lot of effort had gone into building 
the site and students were expected to have taken it more seriously. Incidentally, it also 
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received one of the higher rating on the WIDEC scale but received the lowest student 
ratings on the PURPOSE index iox Instructional Quality (see section 6.7). 

Earlier in section 6.4.2, it was reported that communication with experts was 
limited. Optimal use of the communication functions of the Internet to reach out to 
experts, authors, and peers in a global network was not found in any of the course Web 
sites. The future trend of communication is expected to make use of this capability and it 
is recommended that instructors provide their students with this opportunity and 
encourage them to use these resources. It was reported that students spent most of their 
time on the WWW, searching, downloading and printing. These functions are a waste of 
time unless the exercise is to randomly search the WWW. Why have a course Web site if 
an instructor wants his or her students to do unguided research using the WWW? It's like 
going to the library without a reading list. A course Web site should be designed to 
support students in reducing search time. In the narrative data, students found the course 
Web sites a better resource in comparison to going to the library because they pointed out 
that it cut down on search time. Second, why is downloading and printing rated high on 
the Time Spent dimension? Students should be interacting with the Web site, not 
converting online material to hard copies. Downloading is a bad sign, as it is 
cumbersome if having to read files in various formats (e.g. ADOBE^*^ or OFFICE™), 
time consuming if one has to obtain the necessary software, and costly in terms of 
printing. 

Narrative data from student overall comments highlighted the above. A 
noticeable issue was the organization of the Web site. A well-organized Web site 
appears to facilitate access, saves unnecessary search time, and increases perceived 
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importance. A second issue was efficiency of use, especially related to costs. Students 
in every course commented in one way or the other about the “cost” of using the World 
Wide Web. Many students weighed the costs against the benefits, as did faculty 
members. Three issues dominated the open-ended statements: access, time, and content. 
For example, going to a computer cluster and the time it takes waiting in line for access 
to a computer terminal was weighed against the value of the information that was going 
to be obtained off the Web site. Others weighed the cost of time spent on the Web site in 
relation to the content itself in terms of its importance, relevance, timeliness, and 
currency. Still others weighed cost against the value-added brought about by the 
technology itself in terms of the unique access to specific information, the efficiency of 
downloading/printing, and the time saved in comparison to going to the library, for 
instance. 

What is the solution? Optimize use of WWW multimedia capabilities by: 
providing well organized listings of sites worth visiting so as to reduce uimecessary 
search time; (b) re-creating documents utilizing more interactive formats that are 
engaging, not cognitively taxing, and provide a concise printable version; (c) increasing 
communications by designing the course Web site as a gateway to people networking by 
including listservs, chat rooms, and hypermail as a Web site function; and (d) making 
careful decisions about the purpose of the Web site, the relevance and importance of the 
materials and links. This last issue in particular is discussed further in the next section. 

6.4.2 Practical Implications for Web site Instructional Quality and 
Usefulness 

Earlier in section ??? visual displays emerged as a very useful feature of the Web 
sites. Visual displays were found to reinforce learning and present new information in 
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meaningful ways. Optimal utilization of visual displays emerged as a necessary 
discussion point, rating highly in terms of appeal and presentation of material. The 
capabilities of the course Web sites can be optimized by making appropriate use of 
various multimedia features using meaningful visual displays. Keller (1987a) identified 
three general categories to increase students’ interest and dispel boredom: a) perceptual 
arousal; b) inquiry arousal; and c) variability. Perceptual arousal is a type of curiosity in 
which sudden and unexpected change in the environment activates a person’s attention, 
such as a change in voice level or a surprise piece of information. In Web-based medium, 
animations, blinking text, use of humorous cartoons, and sounds are tools available for 
appropriate use. In the study sample, these tools are important, as evidenced in students’ 
high ratings of visual appeal of Web sites as a mark of quality. This sharply contrasts with 
this authors’ observation of the generally low WIDEC scores on the Attention Gaining 
component of the GAGNE scale. 

Second, opportunities for communicating with faculty and teaching assistants to 
seek clarifications, ask questions, and receive feedback were perceived as most useful. 
These interactive functions represent Gagne’s events pertaining to elicitation of 
information, guided practice, and feedback, and represent some of the seven principles of 
good teaching and learning. Course Web site design could be optimized to include these. 

Third, of equal importance is the clarity of stating the purpose of the course Web 
site and connecting the various functions on the Web site. Similarly, specification of 
Web site functions to course goals and objectives on every given page on the site and not 
just an overall purpose and goal is also important and evidently needs to be improved on 
in this study’s sample of Web sites. Students appreciated being informed about these 
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issues and this seems to have contributed to their perceptions of the importance of the 
course Web sites. 

Fourth, students found visually appealing features of the course Web site to be 
useful in facilitating their recall of prior learning, presenting new information, providing 
learner guidance, and enhancing retention and transfer through enrichment and 
remediation functions. In sum, focusing on interactive, purposeful, and appealing 
functions in the design of course Web sites increases students’ perceptions of their value 
as an instructional resource. 

6.5 Methodological Implications 

Even though the author has made an effort to try to make this study as 
methodologically sound and as generalizable to the intended student population as 
possible, there are some issues that should be discussed at this time. The first one 
involves not accounting for all the variability of the nine course Web sites in terms of 
instructional design, teaching strategies employed, content, context, and purpose. While 
a faculty survey was employed in this study, it took into account only the purpose of the 
course Web sites. Furthermore, all course Web sites, but one, were designed and 
developed by faculty and student assistants without using a shared Web site template or 
consulting with each other on any specific standards. The author was informed that 
within this group, some faculty members used one or two Web sites as models for their 
own. The faculty survey indicated that course Web site design was based on general 
perceived student needs, expectations, and out of an innovative spirit'^. It is important to 



’’ ECN200 faculty questionnaire was not returned therefore no major conclusions are drawn regarding this 
course without faculty perspective. 
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note that, as Clark (1983, 1994) has argued, the use of instructional design as an 
instructional intervention has a larger impact on learning than the technology itself 
While the Web site selection instrument (WIDEC) served to provide some benchmarks 
for making comparisons based on instructional design criteria in this study, the study 
could be extended by taking into account the variables listed above in future research. 

Second, students’ perceptions may have been influenced by their own learning 
styles, instructors' teaching styles and the instructional methodologies applied to the 
course Web site design among others. Already, the narrative data provides some support 
for this contention, as two students indicated that their course Web sites were a reflection 
of their instructor’s teaching style, with its strengths and limitations. This study focused 
on the perceptions of students. It is necessary to acknowledge that perception data is only 
the first sequence of information needed to evaluate a new medium of instructional 
delivery. There are several other areas as noted by Philips (1996) - namely specific 
learning, transfer to other application domains, measurable results and return on 
investment. However, since this was beyond the scope of the study, these issues are not 
discussed but could be controlled for in future studies. 

A third issue involves further instrument validation and reliability testing, 
specifically the Web site Instructional Design Evaluation Checklist (WIDEC) and the 
Student Web site Perception Survey (SWPS) which were developed by the author. 
Replication is necessary to provide further evidence regarding reliability and validity 
under different contexts and with larger sample sizes. While factor loadings of items and 
Cronbach's coefficient Alpha levels were high for SWPS, replication is necessary for 
further establishing the integrity of the instrument. Similarly, while high interclass 
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correlations were obtained for WIDEC, replication of WIDEC and measures of its 
reliability over time, with different groups, are recommended. 

A fourth issue is the actual bias and constraints in the selection of a theoretical 
framework to guide the study. On the one hand, this serves to narrow the scope of the 
study and provides the necessary grounding for study results and findings. On the other 
hand, this limits the authors' epistemological and philosophical approach to the study. 

For example, Gagne’ s work emerged from a behavioristic and cognitivist approach to 
instructional design. Embracing this approach excludes the discussion of other 
theoretical approaches derived from cognitivism, constructivism, and post-modemisim. 
The author acknowledges the importance of contributions made by these alternative 
theoretical perspectives, but limits the study to a focus on systematic instructional design. 
The study contributes to fostering and supporting meaningful learning as cited earlier 

from Khan’s definition (1997); specifically “Web-based instruction (WBI) utilizes the 

attributes and resources of the World Wide Web to create a meaningful learning 
environment where learning is fostered and supported [italics added]” (p. 6). This 
approach was embraced because the goal of the study is to generate findings that focus on 
providing support for the design of instruction in a Web-based environment. Future 
design could utilize other theoretical frameworks to guide elicitation of student 
perceptions to accomplish additional research goals. 

Moreover, the author acknowledges that the survey questions may have focused 
students' responses on a given viewpoint rather than explored their perceptions without 
any prior suppositions. For example, a more constructivist approach to the design of the 
study might have used observations, interviews, and case study methodologies that would 




187 



171 



allow for elicitation of other issues of importance to students that were missed in this 
study. Nonetheless, an argument could be made for any choice of theoretical 
underpinnings made in terms of their relative advantages and disadvantages. Therefore, 
the above constraints were deliberately imposed by the author, as a means for not only 
narrowing the scope and guiding the study, but also because the author believes that the 
theoretical approach to the study lends itself to empirical application in a Web-based 
context. Thus, both the nine events and the seven principles merit testing their validation 
and utility in this new context. In addition, the author believes that the incorporation of 
two theoretical frameworks in one study was too ambitious an effort and recommends 
that future studies focus on further development of the instruments using one framework 
at a time. 

Finally, the extent to which the design of the study served to better explain the 
phenomenon of study is a central issue of this section. The survey served to take a snap 
shot in time of a rapidly changing and growing event. Even the narrative data served the 
same purpose. While the author does not question the choice of methods, she does posit 
the question of how to better study the phenomena given its evolving nature. Cross- 
sectional studies can only provide "snapshots in time" (Pedhazur & Schmelkin, 1991) 
that once taken no longer accurately represent the state of affairs. Longitudinal studies 
may better inform researchers of the capabilities of the medium and its optimal use for 
supporting teaching and learning practices. 
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At the conclusion of this dissertation study, the research has identified several 
areas for future research. These areas will not only help to support or refute findings of 
this study, but they will also assist in furthering the knowledge base of information in this 
area of using the World Wide Web as an instructional resource, be it for campus-based 
traditional education or Web-based instructional design in general. 

1 . Replicate the study with a sample of more sophisticated instructional Web sites that 
rank high on the WIDEC scale and include more of the nine events than this sample. 

2. Narrow scope of study and replicate the dissertation study for each dimension 
separately with a more rigorous design of the student instrument to include more 
items for each dimension. 

3. Conduct further analysis specific to the constructs that emerged from this study. It 
would be interesting to replicate the study with a different student population and 
determine the extent to which similar constructs emerged from the factor analysis. 
Qualitative data collection methods may then be employed to further our 
understanding of why these constructs emerged instead of others. 

4. Extend the development effort of the instruments used in the study WIDEC, SWPS, 
and the faculty survey to generate more rigorous indicators that serve the statistical 
analysis process but also extend the implications for instructional design application 
to Web-based design. WIDEC could be extended to include more Web site functions 
and unique aspects of course Web sites. SPWS could benefit from a more rigorous 
development of items representing the various constructs drawn from the nine events 
and the seven principles of good teaching and learning practices, particularly 
principles that were not well addressed in this study. Identifying more Web site 
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functions leads to further understanding of how to optimize the use of the WWW for 
good teaching and learning practices. The faculty survey could become more 
beneficial if extended to elicit data not only on the purpose of the course Web sites 
but also on the instructors’ teaching strategies, course requirements, and other 
relevant variables. 

5. Conduct a study on the faculty perceptions of the usefulness of course Web sites to 
identify the extent to which this medium is facilitating good teaching practices. 

6. Explore student demographic and characteristics within a given context as possible 
influential factors on students’ perceptions, particularly residential proximity to 
campus, GPA, attitude, and variations among courses for possible insights into 
optimal use of the technology to better meet students' academic needs. 

7. Explore possible influence of field-based and knowledge-based subject areas on 
students’ perceptions of the usefulness of the course Web site. In this study, there is 
evidence that the applied science courses were ranked higher than their art sciences 
counterparts in terms of purpose in particular and overall perceived usefulness as 
supported by the narrative data. 

6.7 Conclusions and Recommendations 

In sum, the World Wide Web is here to stay as evidenced by the increasingly 
prevalent use of WWW technology to further enhance the traditional teaching model in 
mainstream higher education. Further evidence can be found in the ever-growing 
numbers of online courses, course Web sites, and online learning networks. 
Recommendations for instructional design and WWW technology integration efforts 
should take into consideration first and foremost the barriers to use. Access, improved 





174 



Student Internet skills, increased student motivation and efficient and effective use of 
time are crucial to the success of WWW integration efforts. Similarly, improving access 
to the Internet, designing Web sites that make efficient use of students' time on task by 
optimizing the capabilities of the WWW particularly on navigational and organizational 
functions will further enhance integration. Establishing and maintaining conducive and 
facilitative conditions to implement effective teaching and learning practices, from a 
student-users' perspective, necessitates provision of sufficient guidance, the availability 
and accessibility of useful content, and increased facilitation of communication as 
necessary conditions to successful integration. 

The World Wide Web still offers unexplored potential and opportunity to enhance 
learning by becoming a valuable support tool for campus-based courses. This will 
certainly become even more evident as the student population continues to change and 
universities strive to accommodate and adapt to students needs and seek to improve the 
quality of teaching and learning practices using technology as a lever. This chapter has 
provided some guidelines for designing useful course Web sites based on empirical 
findings contributed by the study of nine course Web sites from a range of disciplines, 
used by 142 students representative of Syracuse University students. The findings may 
be generalizable to students at Syracuse University and to students of similar 
demographics and characteristics as SU students studying in U.S. higher education 
institutions. However, the characteristics of the course Web sites and the purposes for 
which they were designed are factors that may not be generalizable as there is 
unaccounted for variance amongst each other and in comparison to other WWW sites. 
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Therefore, this study may in this manner be considered a case study of one large private 
institution’s efforts to integrate WWW technology into the classroom. 

Finally, if one sentence were to capture the major findings of this study it would 
be that a mark of quality and usefulness of a course Web site, from a student perspective, 
is that it is appealing, purposeful, and encourages non mandatory use, free explorations, 
and self-directed learning. The following are the author’s recommendations for 
optimization of using course Web sites as an instructional resource to make them more 
appealing, purposeful, and motivating to students, bringing this report to a close: 

1 . Identify the purpose of your course Web site and convey that purpose to your 
students by relating it to the course purposes and goals. 

2. Specify the purpose and expectations for each segment of your course Web site 
and relate those to the course, unit, or session objectives. 

3. Utilize the visual display capabilities of the medium to increase appeal. 

4. Reduce the need for downloading and printing. 

5. Utilize WWW features to facilitate cognitive learning strategies as suggested. 

6. Utilize multimedia presentations to present material that can take up much of 
class time in note-taking and introductions. This frees up class time for 
discussions, analysis, questions and clarifications of the presentations that 
students reviewed on their course Web site. 

7. Your attitude towards the importance of the course Web site, its relevance to 
class, and its utility as a resource, influences students' perceptions of its quality 
and usefulness, and the overall use of the course Web site. 
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Figure C A Bird’s Eye View of the Study (Courtesy of Gerald Edmonds, PhD.) 








0*5 

cvi 







BEST COPY AVAILABLE 



197 



* 



VITA 

Name of Author: Manal El-Tigi 

Place of Birth: Cairo, Egypt 

Date of Birth: January, 10‘^, 1964 

Graduate and Undergraduate Schools Attended: 

Syracuse University, New York, USA (1994-2000) 

The American University in Cairo, Egypt (1980-1985, 1990-1992) 

Degrees Awarded: 

Master of Science in Instructional Design, Development, and Evaluation, 

Syracuse University, 1996 

Master of Arts in Teaching English as a Foreign Language, 

The American University in Cairo (AUC), 1992 

Bachelor of Arts in Islamic Art and Architecture - Arabic Studies, AUC, 1985 

Awards and Honors: 

Teaching Fellowship, American University in Cairo, 1990-1992 

Dorthea Weinman Scholarship, 1995 
Professional Experience: 

Digital Libraries Assistant - Educational Resource Information Center on Information and 
Technology (ERIC Clearinghouse), Syracuse University, Syracuse, NY. 

Consultant - Office of Professional Development, School of Education, Syracuse University, 
Syracuse, NY. 

Coordinator and Evaluation Specialist - Titus Austin Egypt for Training and Consulting 
Services, Cairo, Egypt. 

Adjunct Faculty - Adult Continuing Education Program - American University in Cairo, Egypt. 





A Pearson’ Product Intercorrelation Matrix for Indexes for Perceived Time Spent, Quality, and Usefulness, Demographics 
and Characteristics (significant at p<0.05) 
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